Model parameters: d_model 2560 ffw_size 10240 kv_size 128 n_heads 20 n_layers 34 Megatron-DeepSpeed/pretrain_gpt.py --tensor-model-parallel-size 1 --pipeline-model-parallel-size 1 --num-layers 34 --hidden-size 2560 --num-attention-heads 20 --kv-channels 128 --ffn-hidden-size 10240 --seq-length 2048 --max-position-embeddings 2048 --micro-batch-size 2 --global-batch-size 512 --train-samples 1 --vocab-file gpt2/vocab.json --merge-file gpt2/merges.txt --clip-grad 1.0 --kill-switch-path kill-switch-2b855b55bc4perplexity50val --bf16 --optimizer adam --adam-beta1 0.9 --adam-beta2 0.999 --adam-eps 1e-8 --lr 2e-4 --min-lr 2e-5 --lr-decay-style cosine --lr-decay-samples 1 --lr-warmup-samples 0 --clip-grad 1.0 --weight-decay 1e-1 --override-lr-scheduler --reset-progress --no-load-optim --log-interval 10 --save-interval 1000 --eval-interval 1 --eval-iters 100 --eval-only true --tensorboard-dir tensorboard_2b855b55bc4perplexity50val --tensorboard-queue-size 5 --log-timers-to-tensorboard --log-batch-size-to-tensorboard --log-validation-ppl-to-tensorboard --save lm1-2b8-55b-c4-perplexity/perplexity50 --load lm1-2b8-55b-c4-perplexity/perplexity50 --train-weighted-split-paths-path train1b5.txt --valid-weighted-split-paths-path val.txt --data-impl mmap --deepspeed --deepspeed_config ds_configs/3511463.json --zero-stage 0 START 3511463: Sat 13 May 2023 12:00:24 PM EEST 0: 0: 0: ======================= ROCm System Management Interface ======================= 0: ================================= Concise Info ================================= 0: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 0: 0 46.0c 100.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 0: 1 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 0: 2 45.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 0: 3 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 0: 4 50.0c 91.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 0: 5 48.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 0: 6 43.0c 87.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 0: 7 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 0: ================================================================================ 0: ============================= End of ROCm SMI Log ============================== 3: 3: 3: ======================= ROCm System Management Interface ======================= 3: ================================= Concise Info ================================= 3: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 3: 0 45.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 3: 1 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 3: 2 41.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 3: 3 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 3: 4 50.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 3: 5 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 3: 6 46.0c 83.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 3: 7 48.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 3: ================================================================================ 3: ============================= End of ROCm SMI Log ============================== 11: 11: 11: ======================= ROCm System Management Interface ======================= 11: ================================= Concise Info ================================= 11: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 11: 0 42.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 11: 1 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 11: 2 42.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 11: 3 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 11: 4 42.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 11: 5 50.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 11: 6 41.0c 93.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 11: 7 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 11: ================================================================================ 11: ============================= End of ROCm SMI Log ============================== 12: 12: 12: ======================= ROCm System Management Interface ======================= 12: ================================= Concise Info ================================= 12: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 12: 0 48.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 12: 1 50.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 12: 2 43.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 12: 3 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 12: 4 46.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 12: 5 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 12: 6 43.0c 95.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 12: 7 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 12: ================================================================================ 12: ============================= End of ROCm SMI Log ============================== 4: 4: 4: ======================= ROCm System Management Interface ======================= 4: ================================= Concise Info ================================= 4: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 4: 0 42.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 4: 1 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 4: 2 38.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 4: 3 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 4: 4 48.0c 91.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 4: 5 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 4: 6 43.0c 96.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 4: 7 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 4: ================================================================================ 4: ============================= End of ROCm SMI Log ============================== 5: 5: 5: ======================= ROCm System Management Interface ======================= 5: ================================= Concise Info ================================= 5: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 5: 0 43.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 5: 1 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 5: 2 43.0c 95.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 5: 3 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 5: 4 44.0c 97.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 5: 5 41.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 5: 6 38.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 5: 7 40.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 5: ================================================================================ 5: ============================= End of ROCm SMI Log ============================== 7: 7: 7: ======================= ROCm System Management Interface ======================= 7: ================================= Concise Info ================================= 7: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 7: 0 43.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 7: 1 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 7: 2 43.0c 96.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 7: 3 49.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 7: 4 41.0c 83.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 7: 5 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 7: 6 42.0c 88.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 7: 7 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 7: ================================================================================ 7: ============================= End of ROCm SMI Log ============================== 16: 16: 16: ======================= ROCm System Management Interface ======================= 16: ================================= Concise Info ================================= 16: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 16: 0 52.0c 93.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 16: 1 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 16: 2 38.0c 87.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 16: 3 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 16: 4 43.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 16: 5 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 16: 6 40.0c 94.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 16: 7 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 16: ================================================================================ 16: ============================= End of ROCm SMI Log ============================== 29: 29: 29: ======================= ROCm System Management Interface ======================= 29: ================================= Concise Info ================================= 29: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 29: 0 45.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 29: 1 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 29: 2 39.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 29: 3 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 29: 4 40.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 29: 5 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 29: 6 38.0c 83.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 29: 7 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 29: ================================================================================ 29: ============================= End of ROCm SMI Log ============================== 22: 22: 22: ======================= ROCm System Management Interface ======================= 22: ================================= Concise Info ================================= 22: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 22: 0 43.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 22: 1 49.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 22: 2 45.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 22: 3 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 22: 4 43.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 22: 5 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 22: 6 42.0c 95.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 22: 7 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 22: ================================================================================ 22: ============================= End of ROCm SMI Log ============================== 28: 28: 28: ======================= ROCm System Management Interface ======================= 28: ================================= Concise Info ================================= 28: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 28: 0 46.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 28: 1 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 28: 2 48.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 28: 3 41.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 28: 4 44.0c 91.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 28: 5 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 28: 6 40.0c 88.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 28: 7 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 28: ================================================================================ 28: ============================= End of ROCm SMI Log ============================== 21: 21: 21: ======================= ROCm System Management Interface ======================= 21: ================================= Concise Info ================================= 21: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 21: 0 44.0c 95.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 21: 1 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 21: 2 41.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 21: 3 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 21: 4 46.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 21: 5 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 21: 6 47.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 21: 7 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 21: ================================================================================ 21: ============================= End of ROCm SMI Log ============================== 9: 9: 9: ======================= ROCm System Management Interface ======================= 9: ================================= Concise Info ================================= 9: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 9: 0 47.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 9: 1 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 9: 2 42.0c 83.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 9: 3 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 9: 4 46.0c 86.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 9: 5 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 9: 6 46.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 9: 7 40.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 9: ================================================================================ 9: ============================= End of ROCm SMI Log ============================== 19: 19: 19: ======================= ROCm System Management Interface ======================= 19: ================================= Concise Info ================================= 19: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 19: 0 46.0c 93.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 19: 1 48.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 19: 2 37.0c 87.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 19: 3 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 19: 4 43.0c 87.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 19: 5 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 19: 6 37.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 19: 7 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 19: ================================================================================ 19: ============================= End of ROCm SMI Log ============================== 15: 15: 15: ======================= ROCm System Management Interface ======================= 15: ================================= Concise Info ================================= 15: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 15: 0 42.0c 94.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 15: 1 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 15: 2 37.0c 93.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 15: 3 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 15: 4 40.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 15: 5 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 15: 6 43.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 15: 7 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 15: ================================================================================ 15: ============================= End of ROCm SMI Log ============================== 31: 31: 31: ======================= ROCm System Management Interface ======================= 31: ================================= Concise Info ================================= 31: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 31: 0 49.0c 84.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 31: 1 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 31: 2 43.0c 86.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 31: 3 41.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 31: 4 42.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 31: 5 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 31: 6 43.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 31: 7 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 31: ================================================================================ 31: ============================= End of ROCm SMI Log ============================== 20: 20: 20: ======================= ROCm System Management Interface ======================= 20: ================================= Concise Info ================================= 20: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 20: 0 45.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 20: 1 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 20: 2 38.0c 97.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 20: 3 40.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 20: 4 44.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 20: 5 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 20: 6 41.0c 84.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 20: 7 49.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 20: ================================================================================ 20: ============================= End of ROCm SMI Log ============================== 27: 27: 27: ======================= ROCm System Management Interface ======================= 27: ================================= Concise Info ================================= 27: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 27: 0 48.0c 87.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 27: 1 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 27: 2 38.0c 88.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 27: 3 41.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 27: 4 41.0c 87.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 27: 5 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 27: 6 42.0c 95.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 27: 7 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 27: ================================================================================ 27: ============================= End of ROCm SMI Log ============================== 13: 13: 13: ======================= ROCm System Management Interface ======================= 13: ================================= Concise Info ================================= 13: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 13: 0 48.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 13: 1 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 13: 2 41.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 13: 3 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 13: 4 47.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 13: 5 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 13: 6 40.0c 93.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 13: 7 39.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 13: ================================================================================ 13: ============================= End of ROCm SMI Log ============================== 30: 30: 30: ======================= ROCm System Management Interface ======================= 30: ================================= Concise Info ================================= 30: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 30: 0 43.0c 88.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 30: 1 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 30: 2 42.0c 88.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 30: 3 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 30: 4 46.0c 95.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 30: 5 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 30: 6 37.0c 88.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 30: 7 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 30: ================================================================================ 30: ============================= End of ROCm SMI Log ============================== 17: 17: 17: ======================= ROCm System Management Interface ======================= 17: ================================= Concise Info ================================= 17: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 17: 0 45.0c 91.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 17: 1 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 17: 2 40.0c 91.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 17: 3 40.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 17: 4 43.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 17: 5 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 17: 6 44.0c 87.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 17: 7 39.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 17: ================================================================================ 17: ============================= End of ROCm SMI Log ============================== 2: 2: 2: ======================= ROCm System Management Interface ======================= 2: ================================= Concise Info ================================= 2: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 2: 0 48.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 2: 1 51.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 2: 2 48.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 2: 3 48.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 2: 4 40.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 2: 5 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 2: 6 41.0c 92.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 2: 7 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 2: ================================================================================ 2: ============================= End of ROCm SMI Log ============================== 23: 23: 23: ======================= ROCm System Management Interface ======================= 23: ================================= Concise Info ================================= 23: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 23: 0 46.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 23: 1 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 23: 2 39.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 23: 3 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 23: 4 40.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 23: 5 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 23: 6 44.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 23: 7 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 23: ================================================================================ 23: ============================= End of ROCm SMI Log ============================== 10: 10: 10: ======================= ROCm System Management Interface ======================= 10: ================================= Concise Info ================================= 10: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 10: 0 46.0c 94.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 10: 1 40.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 10: 2 35.0c 86.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 10: 3 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 10: 4 45.0c 86.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 10: 5 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 10: 6 42.0c 94.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 10: 7 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 10: ================================================================================ 10: ============================= End of ROCm SMI Log ============================== 1: 1: 1: ======================= ROCm System Management Interface ======================= 1: ================================= Concise Info ================================= 1: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 1: 0 43.0c 94.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 1: 1 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 1: 2 40.0c 93.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 1: 3 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 1: 4 47.0c 94.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 1: 5 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 1: 6 41.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 1: 7 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 1: ================================================================================ 1: ============================= End of ROCm SMI Log ============================== 8: 8: 8: ======================= ROCm System Management Interface ======================= 8: ================================= Concise Info ================================= 8: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 8: 0 45.0c 97.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 8: 1 42.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 8: 2 38.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 8: 3 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 8: 4 41.0c 89.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 8: 5 51.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 8: 6 42.0c 86.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 8: 7 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 8: ================================================================================ 8: ============================= End of ROCm SMI Log ============================== 14: 14: 14: ======================= ROCm System Management Interface ======================= 14: ================================= Concise Info ================================= 14: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 14: 0 49.0c 95.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 14: 1 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 14: 2 41.0c 91.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 14: 3 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 14: 4 45.0c 84.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 14: 5 49.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 14: 6 40.0c 83.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 14: 7 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 14: ================================================================================ 14: ============================= End of ROCm SMI Log ============================== 18: 18: 18: ======================= ROCm System Management Interface ======================= 18: ================================= Concise Info ================================= 18: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 18: 0 44.0c 94.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 18: 1 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 18: 2 45.0c 87.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 18: 3 48.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 18: 4 36.0c 88.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 18: 5 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 18: 6 42.0c 86.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 18: 7 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 18: ================================================================================ 18: ============================= End of ROCm SMI Log ============================== 6: 6: 6: ======================= ROCm System Management Interface ======================= 6: ================================= Concise Info ================================= 6: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 6: 0 39.0c 94.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 6: 1 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 6: 2 38.0c 87.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 6: 3 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 6: 4 40.0c 86.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 6: 5 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 6: 6 43.0c 86.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 6: 7 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 6: ================================================================================ 6: ============================= End of ROCm SMI Log ============================== 24: 24: 24: ======================= ROCm System Management Interface ======================= 24: ================================= Concise Info ================================= 24: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 24: 0 45.0c 93.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 24: 1 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 24: 2 43.0c 78.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 24: 3 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 24: 4 44.0c 93.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 24: 5 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 24: 6 43.0c 88.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 24: 7 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 24: ================================================================================ 24: ============================= End of ROCm SMI Log ============================== 26: 26: 26: ======================= ROCm System Management Interface ======================= 26: ================================= Concise Info ================================= 26: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 26: 0 42.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 26: 1 47.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 26: 2 38.0c 95.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 26: 3 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 26: 4 45.0c 86.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 26: 5 46.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 26: 6 39.0c 85.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 26: 7 43.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 26: ================================================================================ 26: ============================= End of ROCm SMI Log ============================== 25: 25: 25: ======================= ROCm System Management Interface ======================= 25: ================================= Concise Info ================================= 25: GPU Temp AvgPwr SCLK MCLK Fan Perf PwrCap VRAM% GPU% 25: 0 45.0c 91.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 25: 1 48.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 25: 2 42.0c 91.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 25: 3 44.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 25: 4 43.0c 88.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 25: 5 50.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 25: 6 44.0c 90.0W 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 25: 7 45.0c N/A 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 25: ================================================================================ 25: ============================= End of ROCm SMI Log ============================== 4: Launching on nid005331 (4/32), master nid005327 port 9999, GPUs 8, CUDA: True 5: Launching on nid005332 (5/32), master nid005327 port 9999, GPUs 8, CUDA: True 2: Launching on nid005329 (2/32), master nid005327 port 9999, GPUs 8, CUDA: True 12: Launching on nid005339 (12/32), master nid005327 port 9999, GPUs 8, CUDA: True 0: Launching on nid005327 (0/32), master nid005327 port 9999, GPUs 8, CUDA: True 30: Launching on nid005933 (30/32), master nid005327 port 9999, GPUs 8, CUDA: True 28: Launching on nid005931 (28/32), master nid005327 port 9999, GPUs 8, CUDA: True 22: Launching on nid005925 (22/32), master nid005327 port 9999, GPUs 8, CUDA: True 29: Launching on nid005932 (29/32), master nid005327 port 9999, GPUs 8, CUDA: True 20: Launching on nid005923 (20/32), master nid005327 port 9999, GPUs 8, CUDA: True 13: Launching on nid005340 (13/32), master nid005327 port 9999, GPUs 8, CUDA: True 11: Launching on nid005338 (11/32), master nid005327 port 9999, GPUs 8, CUDA: True 15: Launching on nid005918 (15/32), master nid005327 port 9999, GPUs 8, CUDA: True 31: Launching on nid005934 (31/32), master nid005327 port 9999, GPUs 8, CUDA: True 27: Launching on nid005930 (27/32), master nid005327 port 9999, GPUs 8, CUDA: True 9: Launching on nid005336 (9/32), master nid005327 port 9999, GPUs 8, CUDA: True 3: Launching on nid005330 (3/32), master nid005327 port 9999, GPUs 8, CUDA: True 7: Launching on nid005334 (7/32), master nid005327 port 9999, GPUs 8, CUDA: True 21: Launching on nid005924 (21/32), master nid005327 port 9999, GPUs 8, CUDA: True 16: Launching on nid005919 (16/32), master nid005327 port 9999, GPUs 8, CUDA: True 17: Launching on nid005920 (17/32), master nid005327 port 9999, GPUs 8, CUDA: True 23: Launching on nid005926 (23/32), master nid005327 port 9999, GPUs 8, CUDA: True 19: Launching on nid005922 (19/32), master nid005327 port 9999, GPUs 8, CUDA: True 24: Launching on nid005927 (24/32), master nid005327 port 9999, GPUs 8, CUDA: True 1: Launching on nid005328 (1/32), master nid005327 port 9999, GPUs 8, CUDA: True 26: Launching on nid005929 (26/32), master nid005327 port 9999, GPUs 8, CUDA: True 25: Launching on nid005928 (25/32), master nid005327 port 9999, GPUs 8, CUDA: True 14: Launching on nid005341 (14/32), master nid005327 port 9999, GPUs 8, CUDA: True 8: Launching on nid005335 (8/32), master nid005327 port 9999, GPUs 8, CUDA: True 10: Launching on nid005337 (10/32), master nid005327 port 9999, GPUs 8, CUDA: True 18: Launching on nid005921 (18/32), master nid005327 port 9999, GPUs 8, CUDA: True 6: Launching on nid005333 (6/32), master nid005327 port 9999, GPUs 8, CUDA: True 0: using world size: 256, data-parallel-size: 256, tensor-model-parallel size: 1, pipeline-model-parallel size: 1 0: accumulate and all-reduce gradients in fp32 for bfloat16 data type. 0: using torch.bfloat16 for parameters ... 0: ------------------------ arguments ------------------------ 0: abort_on_unmet_fused_kernel_constraints ......... False 0: accumulate_allreduce_grads_in_fp32 .............. True 0: adam_beta1 ...................................... 0.9 0: adam_beta2 ...................................... 0.999 0: adam_eps ........................................ 1e-08 0: adlr_autoresume ................................. False 0: adlr_autoresume_interval ........................ 1000 0: apply_query_key_layer_scaling ................... True 0: apply_residual_connection_post_layernorm ........ False 0: attention_dropout ............................... 0.1 0: attention_softmax_in_fp32 ....................... False 0: bert_binary_head ................................ True 0: bert_load ....................................... None 0: bf16 ............................................ True 0: bias_dropout_fusion ............................. True 0: bias_gelu_fusion ................................ True 0: biencoder_projection_dim ........................ 0 0: biencoder_shared_query_context_model ............ False 0: block_data_path ................................. None 0: checkpoint_activations .......................... False 0: checkpoint_in_cpu ............................... False 0: checkpoint_num_layers ........................... 1 0: clip_grad ....................................... 1.0 0: codecarbon_dir .................................. None 0: consumed_train_samples .......................... 0 0: consumed_train_tokens ........................... 0 0: consumed_valid_samples .......................... 0 0: contigious_checkpointing ........................ False 0: cpu_optimizer ................................... False 0: cpu_torch_adam .................................. False 0: curriculum_learning ............................. False 0: data_impl ....................................... mmap 0: data_parallel_size .............................. 256 0: data_path ....................................... None 0: dataloader_type ................................. single 0: DDP_impl ........................................ local 0: decoder_seq_length .............................. None 0: deepscale ....................................... False 0: deepscale_config ................................ None 0: deepspeed ....................................... True 0: deepspeed_activation_checkpointing .............. False 0: deepspeed_config ................................ ds_configs/3511463.json 0: deepspeed_mpi ................................... False 0: distribute_checkpointed_activations ............. False 0: distributed_backend ............................. nccl 0: embed_layernorm ................................. False 0: embedding_path .................................. None 0: encoder_seq_length .............................. 2048 0: eod_mask_loss ................................... False 0: eval_interval ................................... 1 0: eval_iters ...................................... 100 0: eval_only ....................................... True 0: evidence_data_path .............................. None 0: exit_duration_in_mins ........................... None 0: exit_interval ................................... None 0: ffn_hidden_size ................................. 10240 0: finetune ........................................ False 0: fp16 ............................................ False 0: fp16_lm_cross_entropy ........................... False 0: fp32_residual_connection ........................ False 0: gigaflos_no_embeds .............................. 0 0: global_batch_size ............................... 512 0: glu_activation .................................. None 0: hidden_dropout .................................. 0.1 0: hidden_size ..................................... 2560 0: hysteresis ...................................... 2 0: ict_head_size ................................... None 0: ict_load ........................................ None 0: img_dim ......................................... 224 0: indexer_batch_size .............................. 128 0: indexer_log_interval ............................ 1000 0: inference ....................................... False 0: init_method_std ................................. 0.02 0: init_method_xavier_uniform ...................... False 0: initial_loss_scale .............................. 4294967296 0: kill_switch_path ................................ kill-switch-2b855b55bc4perplexity50val 0: kv_channels ..................................... 128 0: layer_norm_fusion ............................... True 0: layernorm_epsilon ............................... 1e-05 0: lazy_mpu_init ................................... None 0: load ............................................ lm1-2b8-55b-c4-perplexity/perplexity50 0: local_rank ...................................... None 0: log_batch_size_to_tensorboard ................... True 0: log_interval .................................... 10 0: log_learning_rate_to_tensorboard ................ True 0: log_level ....................................... None 0: log_level_replica ............................... None 0: log_loss_scale_to_tensorboard ................... True 0: log_num_zeros_in_grad ........................... False 0: log_params_norm ................................. False 0: log_path ........................................ None 0: log_timers_to_tensorboard ....................... True 0: log_validation_ppl_to_tensorboard ............... True 0: loss_on_targets_only ............................ False 0: loss_scale ...................................... None 0: loss_scale_window ............................... 1000 0: lr .............................................. 0.0002 0: lr_decay_iters .................................. None 0: lr_decay_samples ................................ 1 0: lr_decay_style .................................. cosine 0: lr_decay_tokens ................................. None 0: lr_warmup_fraction .............................. None 0: lr_warmup_iters ................................. 0 0: lr_warmup_samples ............................... 0 0: make_vocab_size_divisible_by .................... 128 0: mask_prob ....................................... 0.15 0: masked_softmax_fusion ........................... True 0: max_position_embeddings ......................... 2048 0: mean_noise_span_length .......................... None 0: memory_centric_tiled_linear ..................... False 0: merge_file ...................................... gpt2/merges.txt 0: micro_batch_size ................................ 2 0: min_loss_scale .................................. 1.0 0: min_lr .......................................... 2e-05 0: mmap_warmup ..................................... False 0: no_load_optim ................................... True 0: no_load_rng ..................................... None 0: no_save_optim ................................... None 0: no_save_rng ..................................... None 0: noise_density ................................... None 0: num_attention_heads ............................. 20 0: num_channels .................................... 3 0: num_classes ..................................... 1000 0: num_layers ...................................... 34 0: num_layers_per_virtual_pipeline_stage ........... None 0: num_workers ..................................... 2 0: onnx_safe ....................................... None 0: openai_gelu ..................................... False 0: optimizer ....................................... adam 0: optimizer_fusion ................................ True 0: override_lr_scheduler ........................... True 0: pad_vocab_size_to ............................... None 0: params_dtype .................................... torch.bfloat16 0: partition_activations ........................... False 0: patch_dim ....................................... 16 0: pipeline_model_parallel_size .................... 1 0: position_embedding_type ......................... PositionEmbeddingType.absolute 0: pp_partition_method ............................. None 0: profile_backward ................................ False 0: query_in_block_prob ............................. 0.1 0: rampup_batch_size ............................... None 0: rank ............................................ 0 0: remote_device ................................... none 0: reset_attention_mask ............................ False 0: reset_position_ids .............................. False 0: reset_progress .................................. True 0: retriever_report_topk_accuracies ................ [] 0: retriever_score_scaling ......................... False 0: retriever_seq_length ............................ 256 0: reweight_loss_based_on_position_frequency ....... False 0: sample_rate ..................................... 1.0 0: save ............................................ lm1-2b8-55b-c4-perplexity/perplexity50 0: save_interval ................................... 1000 0: scatter_gather_tensors_in_pipeline .............. True 0: scattered_embeddings ............................ False 0: seed ............................................ 1234 0: seq_length ...................................... 2048 0: sgd_momentum .................................... 0.9 0: short_seq_prob .................................. 0.1 0: skip_train_iteration_range ...................... None 0: split ........................................... None 0: split_transformers .............................. False 0: sync_tp_duplicated_parameters ................... False 0: synchronize_each_layer .......................... False 0: tensor_model_parallel_size ...................... 1 0: tensorboard_dir ................................. tensorboard_2b855b55bc4perplexity50val 0: tensorboard_log_interval ........................ 1 0: tensorboard_queue_size .......................... 5 0: test_weighted_split_paths ....................... None 0: test_weighted_split_paths_path .................. None 0: tile_factor ..................................... 1 0: titles_data_path ................................ None 0: tokenizer_name_or_path .......................... None 0: tokenizer_type .................................. GPT2BPETokenizer 0: train_iters ..................................... None 0: train_samples ................................... 1 0: train_tokens .................................... None 0: train_weighted_split_names ...................... ['train'] 0: train_weighted_split_paths ...................... [['/scratch/project_462000119/data/c4_subsampled/gpt2tok_c4_en_1B5_text_document']] 0: train_weighted_split_paths_path ................. None 0: train_weighted_split_splits ..................... [['0:1']] 0: train_weighted_split_weights .................... [['1.0']] 0: universal_checkpoint ............................ False 0: use_bnb_optimizer ............................... False 0: use_checkpoint_lr_scheduler ..................... False 0: use_contiguous_buffers_in_ddp ................... True 0: use_cpu_initialization .......................... None 0: use_one_sent_docs ............................... False 0: use_pin_memory .................................. False 0: valid_num_workers ............................... 2 0: valid_weighted_split_names ...................... ['validation'] 0: valid_weighted_split_paths ...................... [['/scratch/project_462000119/data/c4_validation/gpt2tok_c4validation_rerun_text_document']] 0: valid_weighted_split_paths_path ................. None 0: valid_weighted_split_splits ..................... [['0:1']] 0: valid_weighted_split_weights .................... [['1.0']] 0: virtual_pipeline_model_parallel_size ............ None 0: vocab_extra_ids ................................. 0 0: vocab_file ...................................... gpt2/vocab.json 0: weight_decay .................................... 0.1 0: world_size ...................................... 256 0: zero_allgather_bucket_size ...................... 0.0 0: zero_contigious_gradients ....................... False 0: zero_reduce_bucket_size ......................... 0.0 0: zero_reduce_scatter ............................. False 0: zero_stage ...................................... 0 0: -------------------- end of arguments --------------------- 0: setting number of micro-batches to constant 1 0: > building GPT2BPETokenizer tokenizer ... 0: > padded vocab (size: 50257) with 47 dummy tokens (new size: 50304) 0: DeepSpeed general environment info: 0: torch install path ............... ['/pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch'] 0: torch version .................... 1.13.0+rocm5.2 0: torch cuda version ............... None 0: torch hip version ................ 5.2.21151-afdc89f8 0: nvcc version ..................... None 0: deepspeed install path ........... ['/pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/deepspeed'] 0: deepspeed info ................... 0.7.5, unknown, unknown 0: deepspeed wheel compiled w. ...... torch 1.13, hip 5.1 0: **** Git info for Megatron: git_hash=unknown git_branch=unknown **** 0: > initializing torch distributed ... 0: [2023-05-13 12:03:30,119] [INFO] [comm.py:633:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl 31: > setting tensorboard ... 0: > initializing tensor model parallel with size 1 0: > initializing pipeline model parallel with size 1 0: > setting random seeds to 1234 ... 0: > initializing model parallel cuda seeds on global rank 0, model parallel rank 0, and data parallel rank 0 with model parallel seed: 3952 and data parallel seed: 1234 0: > compiling dataset index builder ... 0: make: Entering directory '/pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/data' 0: make: Nothing to be done for 'default'. 0: make: Leaving directory '/pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/data' 0: >>> done with dataset index builder. Compilation time: 0.100 seconds 0: > compiling and loading fused kernels ... 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax.cpp -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax_hip.cpp [skipped, already hipified] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax_hip.h [skipped, already hipified] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/compat.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/compat.h [skipped, no changes] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/type_shim.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/type_shim.h [skipped, no changes] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax_cuda.cu -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax_hip.hip [skipped, already hipified] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/type_shim.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/type_shim.h [skipped, no changes] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/compat.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/compat.h [skipped, no changes] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax_hip.h [skipped, already hipified] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax_hip.h [skipped, already hipified] 0: Total number of unsupported CUDA function calls: 0 0: 0: 0: Total number of replaced kernel launches: 87 0: [1/1] c++ scaled_upper_triang_masked_softmax_hip.cuda.o scaled_upper_triang_masked_softmax_hip.o -shared -L/pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/lib -lc10 -lc10_hip -ltorch_cpu -ltorch_hip -ltorch -ltorch_python -L/opt/rocm/lib -lamdhip64 -o scaled_upper_triang_masked_softmax_cuda.so 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax.cpp -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax_hip.cpp [skipped, already hipified] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax_cuda.cu -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax_hip.hip [skipped, already hipified] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/type_shim.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/type_shim.h [skipped, no changes] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/compat.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/compat.h [skipped, no changes] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax_hip.h [skipped, already hipified] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax_hip.h [skipped, already hipified] 0: Total number of unsupported CUDA function calls: 0 0: 0: 0: Total number of replaced kernel launches: 63 0: [1/1] c++ scaled_masked_softmax_hip.cuda.o scaled_masked_softmax_hip.o -shared -L/pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/lib -lc10 -lc10_hip -ltorch_cpu -ltorch_hip -ltorch -ltorch_python -L/opt/rocm/lib -lamdhip64 -o scaled_masked_softmax_cuda.so 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/layer_norm_cuda.cpp -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/layer_norm_cuda.cpp [skipped, no changes] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/layer_norm_cuda_kernel.cu -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/layer_norm_hip_kernel.hip [skipped, already hipified] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/type_shim.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/type_shim.h [skipped, no changes] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/compat.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/compat.h [skipped, no changes] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_upper_triang_masked_softmax_hip.h [skipped, already hipified] 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax.h -> /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/scaled_masked_softmax_hip.h [skipped, already hipified] 0: Total number of unsupported CUDA function calls: 0 0: 0: 0: Total number of replaced kernel launches: 67 0: [1/1] c++ layer_norm_hip_kernel.cuda.o layer_norm_cuda.o -shared -L/pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/lib -lc10 -lc10_hip -ltorch_cpu -ltorch_hip -ltorch -ltorch_python -L/opt/rocm/lib -lamdhip64 -o fused_mix_prec_layer_norm_cuda.so 0: >>> done with compiling and loading fused kernels. Compilation time: 39.873 seconds 0: time to initialize megatron (seconds): 21.266 0: [after megatron is initialized] datetime: 2023-05-13 12:04:21 0: building GPT model ... 0: [2023-05-13 12:04:21,411] [INFO] [utils.py:827:see_memory_usage] Before Building Model 0: [2023-05-13 12:04:21,412] [INFO] [utils.py:828:see_memory_usage] MA 0.0 GB Max_MA 0.0 GB CA 0.0 GB Max_CA 0 GB 0: [2023-05-13 12:04:21,412] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 40.3 GB, percent = 8.0% 0: SEED_LAYERS=False BASE_SEED=1234 SEED_FN=None 0: Using topology: {ProcessCoord(pipe=0, data=0, model=0): 0, ProcessCoord(pipe=0, data=1, model=0): 1, ProcessCoord(pipe=0, data=2, model=0): 2, ProcessCoord(pipe=0, data=3, model=0): 3, ProcessCoord(pipe=0, data=4, model=0): 4, ProcessCoord(pipe=0, data=5, model=0): 5, ProcessCoord(pipe=0, data=6, model=0): 6, ProcessCoord(pipe=0, data=7, model=0): 7, ProcessCoord(pipe=0, data=8, model=0): 8, ProcessCoord(pipe=0, data=9, model=0): 9, ProcessCoord(pipe=0, data=10, model=0): 10, ProcessCoord(pipe=0, data=11, model=0): 11, ProcessCoord(pipe=0, data=12, model=0): 12, ProcessCoord(pipe=0, data=13, model=0): 13, ProcessCoord(pipe=0, data=14, model=0): 14, ProcessCoord(pipe=0, data=15, model=0): 15, ProcessCoord(pipe=0, data=16, model=0): 16, ProcessCoord(pipe=0, data=17, model=0): 17, ProcessCoord(pipe=0, data=18, model=0): 18, ProcessCoord(pipe=0, data=19, model=0): 19, ProcessCoord(pipe=0, data=20, model=0): 20, ProcessCoord(pipe=0, data=21, model=0): 21, ProcessCoord(pipe=0, data=22, model=0): 22, ProcessCoord(pi 0: pe=0, data=23, model=0): 23, ProcessCoord(pipe=0, data=24, model=0): 24, ProcessCoord(pipe=0, data=25, model=0): 25, ProcessCoord(pipe=0, data=26, model=0): 26, ProcessCoord(pipe=0, data=27, model=0): 27, ProcessCoord(pipe=0, data=28, model=0): 28, ProcessCoord(pipe=0, data=29, model=0): 29, ProcessCoord(pipe=0, data=30, model=0): 30, ProcessCoord(pipe=0, data=31, model=0): 31, ProcessCoord(pipe=0, data=32, model=0): 32, ProcessCoord(pipe=0, data=33, model=0): 33, ProcessCoord(pipe=0, data=34, model=0): 34, ProcessCoord(pipe=0, data=35, model=0): 35, ProcessCoord(pipe=0, data=36, model=0): 36, ProcessCoord(pipe=0, data=37, model=0): 37, ProcessCoord(pipe=0, data=38, model=0): 38, ProcessCoord(pipe=0, data=39, model=0): 39, ProcessCoord(pipe=0, data=40, model=0): 40, ProcessCoord(pipe=0, data=41, model=0): 41, ProcessCoord(pipe=0, data=42, model=0): 42, ProcessCoord(pipe=0, data=43, model=0): 43, ProcessCoord(pipe=0, data=44, model=0): 44, ProcessCoord(pipe=0, data=45, model=0): 45, ProcessCoord(pipe=0, data=4 0: 6, model=0): 46, ProcessCoord(pipe=0, data=47, model=0): 47, ProcessCoord(pipe=0, data=48, model=0): 48, ProcessCoord(pipe=0, data=49, model=0): 49, ProcessCoord(pipe=0, data=50, model=0): 50, ProcessCoord(pipe=0, data=51, model=0): 51, ProcessCoord(pipe=0, data=52, model=0): 52, ProcessCoord(pipe=0, data=53, model=0): 53, ProcessCoord(pipe=0, data=54, model=0): 54, ProcessCoord(pipe=0, data=55, model=0): 55, ProcessCoord(pipe=0, data=56, model=0): 56, ProcessCoord(pipe=0, data=57, model=0): 57, ProcessCoord(pipe=0, data=58, model=0): 58, ProcessCoord(pipe=0, data=59, model=0): 59, ProcessCoord(pipe=0, data=60, model=0): 60, ProcessCoord(pipe=0, data=61, model=0): 61, ProcessCoord(pipe=0, data=62, model=0): 62, ProcessCoord(pipe=0, data=63, model=0): 63, ProcessCoord(pipe=0, data=64, model=0): 64, ProcessCoord(pipe=0, data=65, model=0): 65, ProcessCoord(pipe=0, data=66, model=0): 66, ProcessCoord(pipe=0, data=67, model=0): 67, ProcessCoord(pipe=0, data=68, model=0): 68, ProcessCoord(pipe=0, data=69, model=0): 0: 69, ProcessCoord(pipe=0, data=70, model=0): 70, ProcessCoord(pipe=0, data=71, model=0): 71, ProcessCoord(pipe=0, data=72, model=0): 72, ProcessCoord(pipe=0, data=73, model=0): 73, ProcessCoord(pipe=0, data=74, model=0): 74, ProcessCoord(pipe=0, data=75, model=0): 75, ProcessCoord(pipe=0, data=76, model=0): 76, ProcessCoord(pipe=0, data=77, model=0): 77, ProcessCoord(pipe=0, data=78, model=0): 78, ProcessCoord(pipe=0, data=79, model=0): 79, ProcessCoord(pipe=0, data=80, model=0): 80, ProcessCoord(pipe=0, data=81, model=0): 81, ProcessCoord(pipe=0, data=82, model=0): 82, ProcessCoord(pipe=0, data=83, model=0): 83, ProcessCoord(pipe=0, data=84, model=0): 84, ProcessCoord(pipe=0, data=85, model=0): 85, ProcessCoord(pipe=0, data=86, model=0): 86, ProcessCoord(pipe=0, data=87, model=0): 87, ProcessCoord(pipe=0, data=88, model=0): 88, ProcessCoord(pipe=0, data=89, model=0): 89, ProcessCoord(pipe=0, data=90, model=0): 90, ProcessCoord(pipe=0, data=91, model=0): 91, ProcessCoord(pipe=0, data=92, model=0): 92, Process 0: Coord(pipe=0, data=93, model=0): 93, ProcessCoord(pipe=0, data=94, model=0): 94, ProcessCoord(pipe=0, data=95, model=0): 95, ProcessCoord(pipe=0, data=96, model=0): 96, ProcessCoord(pipe=0, data=97, model=0): 97, ProcessCoord(pipe=0, data=98, model=0): 98, ProcessCoord(pipe=0, data=99, model=0): 99, ProcessCoord(pipe=0, data=100, model=0): 100, ProcessCoord(pipe=0, data=101, model=0): 101, ProcessCoord(pipe=0, data=102, model=0): 102, ProcessCoord(pipe=0, data=103, model=0): 103, ProcessCoord(pipe=0, data=104, model=0): 104, ProcessCoord(pipe=0, data=105, model=0): 105, ProcessCoord(pipe=0, data=106, model=0): 106, ProcessCoord(pipe=0, data=107, model=0): 107, ProcessCoord(pipe=0, data=108, model=0): 108, ProcessCoord(pipe=0, data=109, model=0): 109, ProcessCoord(pipe=0, data=110, model=0): 110, ProcessCoord(pipe=0, data=111, model=0): 111, ProcessCoord(pipe=0, data=112, model=0): 112, ProcessCoord(pipe=0, data=113, model=0): 113, ProcessCoord(pipe=0, data=114, model=0): 114, ProcessCoord(pipe=0, data=115, mo 0: del=0): 115, ProcessCoord(pipe=0, data=116, model=0): 116, ProcessCoord(pipe=0, data=117, model=0): 117, ProcessCoord(pipe=0, data=118, model=0): 118, ProcessCoord(pipe=0, data=119, model=0): 119, ProcessCoord(pipe=0, data=120, model=0): 120, ProcessCoord(pipe=0, data=121, model=0): 121, ProcessCoord(pipe=0, data=122, model=0): 122, ProcessCoord(pipe=0, data=123, model=0): 123, ProcessCoord(pipe=0, data=124, model=0): 124, ProcessCoord(pipe=0, data=125, model=0): 125, ProcessCoord(pipe=0, data=126, model=0): 126, ProcessCoord(pipe=0, data=127, model=0): 127, ProcessCoord(pipe=0, data=128, model=0): 128, ProcessCoord(pipe=0, data=129, model=0): 129, ProcessCoord(pipe=0, data=130, model=0): 130, ProcessCoord(pipe=0, data=131, model=0): 131, ProcessCoord(pipe=0, data=132, model=0): 132, ProcessCoord(pipe=0, data=133, model=0): 133, ProcessCoord(pipe=0, data=134, model=0): 134, ProcessCoord(pipe=0, data=135, model=0): 135, ProcessCoord(pipe=0, data=136, model=0): 136, ProcessCoord(pipe=0, data=137, model=0): 137, 0: ProcessCoord(pipe=0, data=138, model=0): 138, ProcessCoord(pipe=0, data=139, model=0): 139, ProcessCoord(pipe=0, data=140, model=0): 140, ProcessCoord(pipe=0, data=141, model=0): 141, ProcessCoord(pipe=0, data=142, model=0): 142, ProcessCoord(pipe=0, data=143, model=0): 143, ProcessCoord(pipe=0, data=144, model=0): 144, ProcessCoord(pipe=0, data=145, model=0): 145, ProcessCoord(pipe=0, data=146, model=0): 146, ProcessCoord(pipe=0, data=147, model=0): 147, ProcessCoord(pipe=0, data=148, model=0): 148, ProcessCoord(pipe=0, data=149, model=0): 149, ProcessCoord(pipe=0, data=150, model=0): 150, ProcessCoord(pipe=0, data=151, model=0): 151, ProcessCoord(pipe=0, data=152, model=0): 152, ProcessCoord(pipe=0, data=153, model=0): 153, ProcessCoord(pipe=0, data=154, model=0): 154, ProcessCoord(pipe=0, data=155, model=0): 155, ProcessCoord(pipe=0, data=156, model=0): 156, ProcessCoord(pipe=0, data=157, model=0): 157, ProcessCoord(pipe=0, data=158, model=0): 158, ProcessCoord(pipe=0, data=159, model=0): 159, ProcessCoor 0: d(pipe=0, data=160, model=0): 160, ProcessCoord(pipe=0, data=161, model=0): 161, ProcessCoord(pipe=0, data=162, model=0): 162, ProcessCoord(pipe=0, data=163, model=0): 163, ProcessCoord(pipe=0, data=164, model=0): 164, ProcessCoord(pipe=0, data=165, model=0): 165, ProcessCoord(pipe=0, data=166, model=0): 166, ProcessCoord(pipe=0, data=167, model=0): 167, ProcessCoord(pipe=0, data=168, model=0): 168, ProcessCoord(pipe=0, data=169, model=0): 169, ProcessCoord(pipe=0, data=170, model=0): 170, ProcessCoord(pipe=0, data=171, model=0): 171, ProcessCoord(pipe=0, data=172, model=0): 172, ProcessCoord(pipe=0, data=173, model=0): 173, ProcessCoord(pipe=0, data=174, model=0): 174, ProcessCoord(pipe=0, data=175, model=0): 175, ProcessCoord(pipe=0, data=176, model=0): 176, ProcessCoord(pipe=0, data=177, model=0): 177, ProcessCoord(pipe=0, data=178, model=0): 178, ProcessCoord(pipe=0, data=179, model=0): 179, ProcessCoord(pipe=0, data=180, model=0): 180, ProcessCoord(pipe=0, data=181, model=0): 181, ProcessCoord(pipe=0, da 0: ta=182, model=0): 182, ProcessCoord(pipe=0, data=183, model=0): 183, ProcessCoord(pipe=0, data=184, model=0): 184, ProcessCoord(pipe=0, data=185, model=0): 185, ProcessCoord(pipe=0, data=186, model=0): 186, ProcessCoord(pipe=0, data=187, model=0): 187, ProcessCoord(pipe=0, data=188, model=0): 188, ProcessCoord(pipe=0, data=189, model=0): 189, ProcessCoord(pipe=0, data=190, model=0): 190, ProcessCoord(pipe=0, data=191, model=0): 191, ProcessCoord(pipe=0, data=192, model=0): 192, ProcessCoord(pipe=0, data=193, model=0): 193, ProcessCoord(pipe=0, data=194, model=0): 194, ProcessCoord(pipe=0, data=195, model=0): 195, ProcessCoord(pipe=0, data=196, model=0): 196, ProcessCoord(pipe=0, data=197, model=0): 197, ProcessCoord(pipe=0, data=198, model=0): 198, ProcessCoord(pipe=0, data=199, model=0): 199, ProcessCoord(pipe=0, data=200, model=0): 200, ProcessCoord(pipe=0, data=201, model=0): 201, ProcessCoord(pipe=0, data=202, model=0): 202, ProcessCoord(pipe=0, data=203, model=0): 203, ProcessCoord(pipe=0, data=204, mode 0: l=0): 204, ProcessCoord(pipe=0, data=205, model=0): 205, ProcessCoord(pipe=0, data=206, model=0): 206, ProcessCoord(pipe=0, data=207, model=0): 207, ProcessCoord(pipe=0, data=208, model=0): 208, ProcessCoord(pipe=0, data=209, model=0): 209, ProcessCoord(pipe=0, data=210, model=0): 210, ProcessCoord(pipe=0, data=211, model=0): 211, ProcessCoord(pipe=0, data=212, model=0): 212, ProcessCoord(pipe=0, data=213, model=0): 213, ProcessCoord(pipe=0, data=214, model=0): 214, ProcessCoord(pipe=0, data=215, model=0): 215, ProcessCoord(pipe=0, data=216, model=0): 216, ProcessCoord(pipe=0, data=217, model=0): 217, ProcessCoord(pipe=0, data=218, model=0): 218, ProcessCoord(pipe=0, data=219, model=0): 219, ProcessCoord(pipe=0, data=220, model=0): 220, ProcessCoord(pipe=0, data=221, model=0): 221, ProcessCoord(pipe=0, data=222, model=0): 222, ProcessCoord(pipe=0, data=223, model=0): 223, ProcessCoord(pipe=0, data=224, model=0): 224, ProcessCoord(pipe=0, data=225, model=0): 225, ProcessCoord(pipe=0, data=226, model=0): 226, P 0: rocessCoord(pipe=0, data=227, model=0): 227, ProcessCoord(pipe=0, data=228, model=0): 228, ProcessCoord(pipe=0, data=229, model=0): 229, ProcessCoord(pipe=0, data=230, model=0): 230, ProcessCoord(pipe=0, data=231, model=0): 231, ProcessCoord(pipe=0, data=232, model=0): 232, ProcessCoord(pipe=0, data=233, model=0): 233, ProcessCoord(pipe=0, data=234, model=0): 234, ProcessCoord(pipe=0, data=235, model=0): 235, ProcessCoord(pipe=0, data=236, model=0): 236, ProcessCoord(pipe=0, data=237, model=0): 237, ProcessCoord(pipe=0, data=238, model=0): 238, ProcessCoord(pipe=0, data=239, model=0): 239, ProcessCoord(pipe=0, data=240, model=0): 240, ProcessCoord(pipe=0, data=241, model=0): 241, ProcessCoord(pipe=0, data=242, model=0): 242, ProcessCoord(pipe=0, data=243, model=0): 243, ProcessCoord(pipe=0, data=244, model=0): 244, ProcessCoord(pipe=0, data=245, model=0): 245, ProcessCoord(pipe=0, data=246, model=0): 246, ProcessCoord(pipe=0, data=247, model=0): 247, ProcessCoord(pipe=0, data=248, model=0): 248, ProcessCoord( 0: pipe=0, data=249, model=0): 249, ProcessCoord(pipe=0, data=250, model=0): 250, ProcessCoord(pipe=0, data=251, model=0): 251, ProcessCoord(pipe=0, data=252, model=0): 252, ProcessCoord(pipe=0, data=253, model=0): 253, ProcessCoord(pipe=0, data=254, model=0): 254, ProcessCoord(pipe=0, data=255, model=0): 255} 0: [2023-05-13 12:04:29,607] [INFO] [module.py:366:_partition_layers] Partitioning pipeline stages with method type:transformer 0: stage=0 layers=41 0: 0: _to_float16 0: 1: EmbeddingPipe 0: 2: 0: 3: ParallelTransformerLayerPipe 0: 4: ParallelTransformerLayerPipe 0: 5: ParallelTransformerLayerPipe 0: 6: ParallelTransformerLayerPipe 0: 7: ParallelTransformerLayerPipe 0: 8: ParallelTransformerLayerPipe 0: 9: ParallelTransformerLayerPipe 0: 10: ParallelTransformerLayerPipe 0: 11: ParallelTransformerLayerPipe 0: 12: ParallelTransformerLayerPipe 0: 13: ParallelTransformerLayerPipe 0: 14: ParallelTransformerLayerPipe 0: 15: ParallelTransformerLayerPipe 0: 16: ParallelTransformerLayerPipe 0: 17: ParallelTransformerLayerPipe 0: 18: ParallelTransformerLayerPipe 0: 19: ParallelTransformerLayerPipe 0: 20: ParallelTransformerLayerPipe 0: 21: ParallelTransformerLayerPipe 0: 22: ParallelTransformerLayerPipe 0: 23: ParallelTransformerLayerPipe 0: 24: ParallelTransformerLayerPipe 0: 25: ParallelTransformerLayerPipe 0: 26: ParallelTransformerLayerPipe 0: 27: ParallelTransformerLayerPipe 0: 28: ParallelTransformerLayerPipe 0: 29: ParallelTransformerLayerPipe 0: 30: ParallelTransformerLayerPipe 0: 31: ParallelTransformerLayerPipe 0: 32: ParallelTransformerLayerPipe 0: 33: ParallelTransformerLayerPipe 0: 34: ParallelTransformerLayerPipe 0: 35: ParallelTransformerLayerPipe 0: 36: ParallelTransformerLayerPipe 0: 37: undo 0: 38: MixedFusedLayerNorm 0: 39: EmbeddingPipe 0: 40: float16_to_fp32 0: loss: CrossEntropy 0: [2023-05-13 12:04:30,044] [INFO] [utils.py:827:see_memory_usage] After Building Model 0: [2023-05-13 12:04:30,044] [INFO] [utils.py:828:see_memory_usage] MA 5.26 GB Max_MA 5.26 GB CA 5.31 GB Max_CA 5 GB 0: [2023-05-13 12:04:30,044] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 40.48 GB, percent = 8.0% 0: setting training iterations to 0 0: > learning rate decay style: cosine 0: DeepSpeed is enabled. 0: [2023-05-13 12:04:30,047] [INFO] [logging.py:68:log_dist] [Rank 0] DeepSpeed info: version=0.7.5, git-hash=unknown, git-branch=unknown 0: [2023-05-13 12:04:53,471] [INFO] [logging.py:68:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False 0: [2023-05-13 12:04:53,471] [INFO] [logging.py:68:log_dist] [Rank 0] Removing param_group that has no 'params' in the client Optimizer 0: [2023-05-13 12:04:53,471] [INFO] [logging.py:68:log_dist] [Rank 0] Using client Optimizer as basic optimizer 0: [2023-05-13 12:04:53,490] [INFO] [logging.py:68:log_dist] [Rank 0] DeepSpeed Basic Optimizer = FusedAdam 0: [2023-05-13 12:04:53,490] [INFO] [logging.py:68:log_dist] [Rank 0] Creating BF16 optimizer 0: [2023-05-13 12:04:53,616] [INFO] [utils.py:827:see_memory_usage] begin bf16_optimizer 0: [2023-05-13 12:04:53,617] [INFO] [utils.py:828:see_memory_usage] MA 5.25 GB Max_MA 5.27 GB CA 5.32 GB Max_CA 5 GB 0: [2023-05-13 12:04:53,617] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.14 GB, percent = 8.2% 1: ninja: no work to do. 16: Time to load utils op: 0.4856691360473633 seconds 16: Time to load utils op: 0.4855477809906006 secondsTime to load utils op: 0.4865686893463135 seconds 16: 16: Time to load utils op: 0.48536038398742676 seconds 16: Time to load utils op: 0.4866347312927246 seconds 16: Time to load utils op: 0.48665452003479004 seconds 16: Time to load utils op: 0.48546791076660156 secondsTime to load utils op: 0.48637890815734863 seconds 16: 27: Time to load utils op: 0.46270203590393066 seconds 27: Time to load utils op: 0.46274352073669434 seconds 27: Time to load utils op: 0.4627361297607422 secondsTime to load utils op: 0.46274423599243164 seconds 27: 27: Time to load utils op: 0.46277618408203125 secondsTime to load utils op: 0.4627656936645508 seconds 27: 27: Time to load utils op: 0.46277856826782227 seconds 27: Time to load utils op: 0.4627854824066162 seconds 11: Time to load utils op: 0.5175433158874512 secondsTime to load utils op: 0.5178415775299072 seconds 11: 11: Time to load utils op: 0.5175004005432129 seconds 11: Time to load utils op: 0.5171833038330078 seconds 11: Time to load utils op: 0.5174016952514648 seconds 11: Time to load utils op: 0.5182991027832031 seconds 11: Time to load utils op: 0.5183403491973877 seconds 11: Time to load utils op: 0.5180737972259521 seconds 0: Time to load utils op: 0.5344240665435791 seconds 0: Time to load utils op: 0.534548282623291 seconds 31: Time to load utils op: 0.4966099262237549 secondsTime to load utils op: 0.4957132339477539 secondsTime to load utils op: 0.49622273445129395 seconds 31: 31: 31: Time to load utils op: 0.4955289363861084 seconds 31: Time to load utils op: 0.49649882316589355 secondsTime to load utils op: 0.49689173698425293 seconds 31: 0: Time to load utils op: 0.4898085594177246 seconds 0: Time to load utils op: 0.5345780849456787 seconds 0: Time to load utils op: 0.5339996814727783 seconds 0: Time to load utils op: 0.5347683429718018 secondsTime to load utils op: 0.5337302684783936 seconds 0: 0: Time to load utils op: 0.534095287322998 seconds 31: Time to load utils op: 0.49661731719970703 secondsTime to load utils op: 0.4951615333557129 seconds 31: 10: Time to load utils op: 0.4761636257171631 seconds 10: Time to load utils op: 0.4761795997619629 seconds 1: Time to load utils op: 0.5342583656311035 seconds 1: Time to load utils op: 0.532989501953125 seconds 10: Time to load utils op: 0.4762229919433594 seconds 10: Time to load utils op: 0.47622013092041016 seconds 10: Time to load utils op: 0.4762382507324219 seconds 9: Time to load utils op: 0.4757692813873291 seconds 9: Time to load utils op: 0.47579169273376465 seconds 10: Time to load utils op: 0.47624778747558594 secondsTime to load utils op: 0.4762554168701172 seconds 10: 1: Time to load utils op: 0.5349729061126709 secondsTime to load utils op: 0.5343024730682373 seconds 1: 10: Time to load utils op: 0.4762387275695801 seconds 1: Time to load utils op: 0.5332772731781006 secondsTime to load utils op: 0.5342259407043457 seconds 1: 1: Time to load utils op: 0.5341470241546631 seconds 9: Time to load utils op: 0.4757826328277588 seconds 1: Time to load utils op: 0.5330398082733154 seconds 9: Time to load utils op: 0.4758014678955078 seconds 9: Time to load utils op: 0.4758331775665283 seconds 22: Time to load utils op: 0.4757347106933594 seconds 22: Time to load utils op: 0.4757215976715088 seconds 22: Time to load utils op: 0.47576355934143066 seconds 9: Time to load utils op: 0.475818395614624 seconds 13: Time to load utils op: 0.4757976531982422 seconds 13: Time to load utils op: 0.4757397174835205 seconds 22: Time to load utils op: 0.47579216957092285 seconds 9: Time to load utils op: 0.4758474826812744 seconds 22: Time to load utils op: 0.47577786445617676 secondsTime to load utils op: 0.47580838203430176 seconds 22: Time to load utils op: 0.4757657051086426 seconds 22: 9: Time to load utils op: 0.475844144821167 seconds 22: Time to load utils op: 0.4757857322692871 seconds 13: Time to load utils op: 0.4758298397064209 secondsTime to load utils op: 0.4758317470550537 seconds 13: 13: Time to load utils op: 0.4758424758911133 seconds 13: Time to load utils op: 0.4758491516113281 secondsTime to load utils op: 0.47589921951293945 seconds 13: 13: Time to load utils op: 0.4758579730987549 seconds 24: Time to load utils op: 0.4759483337402344 secondsTime to load utils op: 0.4759519100189209 seconds 24: 24: Time to load utils op: 0.47596096992492676 seconds 24: Time to load utils op: 0.47598791122436523 seconds 24: Time to load utils op: 0.47599291801452637 seconds 24: Time to load utils op: 0.4760141372680664 secondsTime to load utils op: 0.4760010242462158 seconds 24: 24: Time to load utils op: 0.47603821754455566 seconds 20: Time to load utils op: 0.4762589931488037 secondsTime to load utils op: 0.4762256145477295 seconds 20: 6: Time to load utils op: 0.5276808738708496 seconds 6: Time to load utils op: 0.5284113883972168 seconds 20: Time to load utils op: 0.47624659538269043 seconds 20: Time to load utils op: 0.4762413501739502 seconds 20: Time to load utils op: 0.4762566089630127 seconds 3: Time to load utils op: 0.47643303871154785 seconds 3: Time to load utils op: 0.47647905349731445 secondsTime to load utils op: 0.47647595405578613 seconds 3: 3: Time to load utils op: 0.4765043258666992 seconds 6: Time to load utils op: 0.5277631282806396 seconds 6: Time to load utils op: 0.5277657508850098 seconds 20: Time to load utils op: 0.4762263298034668 seconds 6: Time to load utils op: 0.5277767181396484 seconds 20: Time to load utils op: 0.4763145446777344 secondsTime to load utils op: 0.47627878189086914 seconds 20: 3: Time to load utils op: 0.47647953033447266 seconds 3: Time to load utils op: 0.47650742530822754 secondsTime to load utils op: 0.4764847755432129 seconds 3: 6: Time to load utils op: 0.5286126136779785 seconds 28: Time to load utils op: 0.47646212577819824 secondsTime to load utils op: 0.47646665573120117 seconds 28: 28: Time to load utils op: 0.4764697551727295 seconds 28: Time to load utils op: 0.476438045501709 secondsTime to load utils op: 0.4764726161956787 seconds 28: Time to load utils op: 0.4764840602874756 seconds 2: Time to load utils op: 0.5346250534057617 seconds 3: Time to load utils op: 0.4765186309814453 seconds 6: Time to load utils op: 0.5285754203796387 secondsTime to load utils op: 0.527759313583374 seconds 6: 15: Time to load utils op: 0.5168359279632568 secondsTime to load utils op: 0.5169928073883057 seconds 15: 7: Time to load utils op: 0.4765594005584717 secondsTime to load utils op: 0.47646570205688477 seconds 7: 7: Time to load utils op: 0.47658658027648926 seconds 7: Time to load utils op: 0.47658300399780273 seconds 28: 28: Time to load utils op: 0.476468563079834 secondsTime to load utils op: 0.4765040874481201 seconds 28: 2: Time to load utils op: 0.5338852405548096 seconds 2: Time to load utils op: 0.534759521484375 seconds 2: Time to load utils op: 0.5336017608642578 seconds 2: Time to load utils op: 0.534895658493042 seconds 2: Time to load utils op: 0.5336470603942871 seconds 2: Time to load utils op: 0.5339627265930176 seconds 7: Time to load utils op: 0.4766428470611572 secondsTime to load utils op: 0.4766089916229248 seconds 7: 15: Time to load utils op: 0.5179173946380615 secondsTime to load utils op: 0.5186569690704346 seconds 15: 15: Time to load utils op: 0.5179345607757568 seconds 15: Time to load utils op: 0.5173180103302002 seconds 2: Time to load utils op: 0.5343983173370361 seconds 4: Time to load utils op: 0.5316464900970459 secondsTime to load utils op: 0.5323376655578613 secondsTime to load utils op: 0.5323505401611328 secondsTime to load utils op: 0.5313248634338379 seconds 4: 4: 4: 7: Time to load utils op: 0.47664546966552734 secondsTime to load utils op: 0.476642370223999 seconds 7: 15: Time to load utils op: 0.5163493156433105 secondsTime to load utils op: 0.5173296928405762 seconds 15: 30: Time to load utils op: 0.4766557216644287 seconds 30: Time to load utils op: 0.4766852855682373 seconds 30: Time to load utils op: 0.4766836166381836 seconds 21: Time to load utils op: 0.5105190277099609 seconds 21: Time to load utils op: 0.5104978084564209 seconds 4: Time to load utils op: 0.5313148498535156 seconds 30: Time to load utils op: 0.476686954498291 seconds 4: Time to load utils op: 0.5323009490966797 secondsTime to load utils op: 0.5319297313690186 secondsTime to load utils op: 0.5313723087310791 seconds 4: 4: 30: Time to load utils op: 0.47671031951904297 secondsTime to load utils op: 0.47670745849609375 seconds 30: 30: Time to load utils op: 0.4767003059387207 secondsTime to load utils op: 0.4767289161682129 seconds 30: 8: Time to load utils op: 0.47698497772216797 seconds 8: Time to load utils op: 0.47700071334838867 seconds 18: Time to load utils op: 0.5120298862457275 seconds 18: Time to load utils op: 0.5128347873687744 seconds 21: Time to load utils op: 0.5097267627716064 seconds 21: Time to load utils op: 0.5109224319458008 secondsTime to load utils op: 0.5097610950469971 secondsTime to load utils op: 0.5099823474884033 seconds 21: 21: 21: Time to load utils op: 0.5105478763580322 seconds 21: Time to load utils op: 0.5096790790557861 seconds 8: Time to load utils op: 0.4770243167877197 seconds 8: Time to load utils op: 0.477031946182251 secondsTime to load utils op: 0.4770331382751465 seconds 8: 18: Time to load utils op: 0.5134224891662598 secondsTime to load utils op: 0.5135085582733154 seconds 18: 18: Time to load utils op: 0.5133969783782959 seconds 18: Time to load utils op: 0.513444185256958 secondsTime to load utils op: 0.511315107345581 seconds 8: Time to load utils op: 0.47704410552978516 seconds 8: Time to load utils op: 0.4770476818084717 seconds 18: 18: Time to load utils op: 0.5134103298187256 seconds 25: Time to load utils op: 0.47692060470581055 seconds 25: Time to load utils op: 0.4769325256347656 seconds 8: Time to load utils op: 0.4770967960357666 seconds 25: Time to load utils op: 0.47694969177246094 seconds 25: Time to load utils op: 0.47698307037353516 seconds 25: Time to load utils op: 0.477001428604126 secondsTime to load utils op: 0.4770052433013916 seconds 25: 25: Time to load utils op: 0.47701430320739746 secondsTime to load utils op: 0.47701525688171387 seconds 25: 26: Time to load utils op: 0.5154876708984375 secondsTime to load utils op: 0.5154449939727783 seconds 26: 26: Time to load utils op: 0.5154457092285156 secondsTime to load utils op: 0.515357255935669 seconds 26: 26: Time to load utils op: 0.515268087387085 seconds 26: Time to load utils op: 0.5156269073486328 seconds 26: Time to load utils op: 0.5154569149017334 seconds 26: Time to load utils op: 0.5154776573181152 seconds 29: Time to load utils op: 0.519465446472168 seconds 29: Time to load utils op: 0.5182027816772461 seconds 29: Time to load utils op: 0.5189027786254883 seconds 29: Time to load utils op: 0.5195121765136719 seconds 29: Time to load utils op: 0.5186340808868408 seconds 29: Time to load utils op: 0.5199437141418457 seconds 29: Time to load utils op: 0.5190799236297607 seconds 29: Time to load utils op: 0.5191829204559326 seconds 5: Time to load utils op: 0.559882640838623 seconds 5: Time to load utils op: 0.5615975856781006 secondsTime to load utils op: 0.5600800514221191 seconds 5: 5: Time to load utils op: 0.560692310333252 seconds 5: Time to load utils op: 0.5607452392578125 seconds 5: Time to load utils op: 0.5613601207733154 secondsTime to load utils op: 0.5602500438690186 seconds 5: 5: Time to load utils op: 0.5602817535400391 seconds 14: Time to load utils op: 0.5537168979644775 seconds 14: Time to load utils op: 0.5544753074645996 seconds 14: Time to load utils op: 0.5541708469390869 seconds 14: Time to load utils op: 0.5537035465240479 secondsTime to load utils op: 0.5542821884155273 secondsTime to load utils op: 0.5549771785736084 seconds 14: 14: 14: Time to load utils op: 0.5551393032073975 seconds 14: Time to load utils op: 0.5539004802703857 seconds 19: Time to load utils op: 0.5567517280578613 seconds 19: Time to load utils op: 0.5567948818206787 seconds 19: Time to load utils op: 0.557410717010498 seconds 19: Time to load utils op: 0.5571744441986084 seconds 19: Time to load utils op: 0.5579628944396973 seconds 19: Time to load utils op: 0.557551383972168 secondsTime to load utils op: 0.558361291885376 seconds 19: 19: Time to load utils op: 0.557253360748291 seconds 12: Time to load utils op: 0.5979163646697998 seconds 12: Time to load utils op: 0.5981600284576416 seconds 12: Time to load utils op: 0.5986266136169434 seconds 12: Time to load utils op: 0.5988593101501465 seconds 12: Time to load utils op: 0.5980947017669678 seconds 12: Time to load utils op: 0.5989937782287598 seconds 12: Time to load utils op: 0.5981214046478271 seconds 12: Time to load utils op: 0.5985727310180664 seconds 0: Time to load utils op: 0.0005245208740234375 seconds 0: Time to load utils op: 0.0005040168762207031 seconds 0: Time to load utils op: 0.0005018711090087891 seconds 0: Time to load utils op: 0.00045228004455566406 seconds 0: Time to load utils op: 0.0004534721374511719 seconds 0: Time to load utils op: 0.0004191398620605469 seconds 0: Time to load utils op: 0.00040650367736816406 seconds 15: Time to load utils op: 0.0008721351623535156 seconds 15: Time to load utils op: 0.0009987354278564453 seconds 2: Time to load utils op: 0.0012784004211425781 seconds 2: Time to load utils op: 0.0013022422790527344 seconds 11: Time to load utils op: 0.0009062290191650391 seconds 11: Time to load utils op: 0.000804901123046875 seconds 2: Time to load utils op: 0.0014247894287109375 secondsTime to load utils op: 0.0014085769653320312 seconds 2: 2: Time to load utils op: 0.0014564990997314453 seconds 2: Time to load utils op: 0.001413583755493164 seconds 15: Time to load utils op: 0.0013687610626220703 seconds 2: Time to load utils op: 0.0010502338409423828 seconds 11: Time to load utils op: 0.001096487045288086 seconds 15: Time to load utils op: 0.0013647079467773438 seconds 2: Time to load utils op: 0.0014793872833251953 seconds 15: Time to load utils op: 0.0012679100036621094 seconds 15: Time to load utils op: 0.001257181167602539 secondsTime to load utils op: 0.0012865066528320312 seconds 15: 15: Time to load utils op: 0.0012896060943603516 seconds 11: Time to load utils op: 0.0012884140014648438 seconds 11: Time to load utils op: 0.0013082027435302734 seconds 11: Time to load utils op: 0.0012269020080566406 seconds 11: Time to load utils op: 0.0012390613555908203 seconds 11: Time to load utils op: 0.0013222694396972656 seconds 21: Time to load utils op: 0.0009679794311523438 seconds 14: Time to load utils op: 0.0010805130004882812 secondsTime to load utils op: 0.0010225772857666016 secondsTime to load utils op: 0.0010712146759033203 seconds 14: 14: 21: Time to load utils op: 0.0012805461883544922 seconds 21: Time to load utils op: 0.0012543201446533203 seconds 19: Time to load utils op: 0.0011515617370605469 seconds 21: Time to load utils op: 0.001262664794921875 seconds 14: Time to load utils op: 0.0013012886047363281 seconds 14: Time to load utils op: 0.001346588134765625 seconds 21: Time to load utils op: 0.0013775825500488281 seconds 1: Time to load utils op: 0.000988006591796875 seconds 14: Time to load utils op: 0.0013339519500732422 seconds 21: Time to load utils op: 0.0012946128845214844 seconds 21: Time to load utils op: 0.0012927055358886719 seconds 14: Time to load utils op: 0.0012385845184326172 seconds 1: Time to load utils op: 0.000782012939453125 seconds 14: Time to load utils op: 0.0012969970703125 seconds 21: Time to load utils op: 0.0013501644134521484 seconds 19: Time to load utils op: 0.0014390945434570312 seconds 1: Time to load utils op: 0.0012729167938232422 seconds 19: Time to load utils op: 0.0015676021575927734 seconds 19: Time to load utils op: 0.0015835762023925781 seconds 1: Time to load utils op: 0.001373291015625 secondsTime to load utils op: 0.0012774467468261719 seconds 1: 19: Time to load utils op: 0.0015482902526855469 seconds 19: Time to load utils op: 0.0015673637390136719 seconds 19: Time to load utils op: 0.0014755725860595703 seconds 1: Time to load utils op: 0.0013859272003173828 seconds 1: Time to load utils op: 0.0013713836669921875 seconds 19: Time to load utils op: 0.0016262531280517578 seconds 1: Time to load utils op: 0.0014181137084960938 seconds 16: Time to load utils op: 0.0009927749633789062 seconds 16: Time to load utils op: 0.0011107921600341797 seconds 18: Time to load utils op: 0.001123189926147461 seconds 18: Time to load utils op: 0.0010197162628173828 seconds 18: Time to load utils op: 0.001314401626586914 seconds 18: Time to load utils op: 0.001329660415649414 seconds 16: Time to load utils op: 0.0014712810516357422 seconds 16: Time to load utils op: 0.001346588134765625 secondsTime to load utils op: 0.0014700889587402344 seconds 16: 18: Time to load utils op: 0.0013036727905273438 seconds 18: Time to load utils op: 0.0013167858123779297 seconds 18: Time to load utils op: 0.001285552978515625 seconds 16: Time to load utils op: 0.0013573169708251953 seconds 18: Time to load utils op: 0.0013422966003417969 seconds 16: Time to load utils op: 0.0013566017150878906 seconds 16: Time to load utils op: 0.0013973712921142578 seconds 23: Time to load utils op: 0.6261794567108154 seconds 23: Time to load utils op: 0.6261351108551025 seconds 23: Time to load utils op: 0.6252853870391846 secondsTime to load utils op: 0.6248581409454346 secondsTime to load utils op: 0.625237226486206 seconds 23: 23: 23: Time to load utils op: 0.6253049373626709 seconds 23: Time to load utils op: 0.6241092681884766 seconds 23: Time to load utils op: 0.6262202262878418 seconds 17: Time to load utils op: 0.6343598365783691 seconds 17: Time to load utils op: 0.6351690292358398 secondsTime to load utils op: 0.635549783706665 seconds 17: Time to load utils op: 0.6355149745941162 seconds 17: 17: Time to load utils op: 0.6346521377563477 seconds 17: Time to load utils op: 0.6351432800292969 secondsTime to load utils op: 0.6359765529632568 seconds 17: 17: Time to load utils op: 0.6354398727416992 seconds 5: Time to load utils op: 0.0007069110870361328 seconds 5: Time to load utils op: 0.0006628036499023438 seconds 5: Time to load utils op: 0.0010890960693359375 seconds 5: Time to load utils op: 0.0011584758758544922 secondsTime to load utils op: 0.0011723041534423828 seconds 5: 5: Time to load utils op: 0.0010745525360107422 seconds 5: Time to load utils op: 0.0011322498321533203 seconds 5: Time to load utils op: 0.0011377334594726562 seconds 0: [2023-05-13 12:04:54,236] [INFO] [utils.py:827:see_memory_usage] before initializing group 0 0: [2023-05-13 12:04:54,237] [INFO] [utils.py:828:see_memory_usage] MA 5.25 GB Max_MA 5.25 GB CA 5.32 GB Max_CA 5 GB 0: [2023-05-13 12:04:54,237] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.17 GB, percent = 8.2% 23: Time to load utils op: 0.0006792545318603516 seconds 23: Time to load utils op: 0.0006563663482666016 seconds 23: Time to load utils op: 0.0007114410400390625 seconds 23: Time to load utils op: 0.0006928443908691406 secondsTime to load utils op: 0.0006861686706542969 seconds 23: 23: Time to load utils op: 0.0007069110870361328 seconds 23: Time to load utils op: 0.0007159709930419922 seconds 23: Time to load utils op: 0.0008757114410400391 seconds 0: [2023-05-13 12:04:54,368] [INFO] [utils.py:827:see_memory_usage] after initializing group 0 0: [2023-05-13 12:04:54,369] [INFO] [utils.py:828:see_memory_usage] MA 10.64 GB Max_MA 10.64 GB CA 13.39 GB Max_CA 13 GB 0: [2023-05-13 12:04:54,369] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.33 GB, percent = 8.2% 0: [2023-05-13 12:04:54,479] [INFO] [utils.py:827:see_memory_usage] before initializing group 1 0: [2023-05-13 12:04:54,479] [INFO] [utils.py:828:see_memory_usage] MA 10.64 GB Max_MA 10.64 GB CA 13.39 GB Max_CA 13 GB 0: [2023-05-13 12:04:54,479] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.33 GB, percent = 8.2% 0: [2023-05-13 12:04:54,588] [INFO] [utils.py:827:see_memory_usage] after initializing group 1 0: [2023-05-13 12:04:54,589] [INFO] [utils.py:828:see_memory_usage] MA 15.73 GB Max_MA 15.73 GB CA 21.01 GB Max_CA 21 GB 0: [2023-05-13 12:04:54,589] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.33 GB, percent = 8.2% 0: [2023-05-13 12:04:54,693] [INFO] [utils.py:827:see_memory_usage] before initializing group 2 0: [2023-05-13 12:04:54,694] [INFO] [utils.py:828:see_memory_usage] MA 15.73 GB Max_MA 15.73 GB CA 21.01 GB Max_CA 21 GB 0: [2023-05-13 12:04:54,694] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.33 GB, percent = 8.2% 0: [2023-05-13 12:04:54,803] [INFO] [utils.py:827:see_memory_usage] after initializing group 2 0: [2023-05-13 12:04:54,803] [INFO] [utils.py:828:see_memory_usage] MA 15.74 GB Max_MA 15.74 GB CA 21.01 GB Max_CA 21 GB 0: [2023-05-13 12:04:54,804] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.33 GB, percent = 8.2% 0: [2023-05-13 12:04:54,907] [INFO] [utils.py:827:see_memory_usage] before initialize_optimizer 0: [2023-05-13 12:04:54,908] [INFO] [utils.py:828:see_memory_usage] MA 15.74 GB Max_MA 15.74 GB CA 21.01 GB Max_CA 21 GB 0: [2023-05-13 12:04:54,908] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.33 GB, percent = 8.2% 31: Time to load utils op: 0.0012052059173583984 seconds 4: Time to load utils op: 0.0012009143829345703 seconds 31: Time to load utils op: 0.0017442703247070312 seconds 27: Time to load utils op: 0.0013048648834228516 seconds 12: Time to load utils op: 0.0013377666473388672 seconds 12: Time to load utils op: 0.0013153553009033203 seconds 12: Time to load utils op: 0.0013728141784667969 seconds 12: Time to load utils op: 0.0013668537139892578 seconds 10: Time to load utils op: 0.0012083053588867188 seconds 12: Time to load utils op: 0.0013628005981445312 seconds 12: Time to load utils op: 0.0014185905456542969 seconds 3: Time to load utils op: 0.0018210411071777344 seconds 4: Time to load utils op: 0.0016906261444091797 seconds 12: Time to load utils op: 0.0016090869903564453 seconds 28: Time to load utils op: 0.0016450881958007812 seconds 3: Time to load utils op: 0.0018627643585205078 seconds 3: Time to load utils op: 0.001882314682006836 seconds 8: Time to load utils op: 0.0016374588012695312 seconds 10: Time to load utils op: 0.0014493465423583984 seconds 8: Time to load utils op: 0.0017323493957519531 seconds 13: Time to load utils op: 0.0011777877807617188 seconds 29: Time to load utils op: 0.0016252994537353516 seconds 30: Time to load utils op: 0.0014348030090332031 seconds 12: Time to load utils op: 0.002026081085205078 seconds 28: Time to load utils op: 0.001737833023071289 seconds 30: Time to load utils op: 0.0019338130950927734 seconds 22: Time to load utils op: 0.0018665790557861328 seconds 10: Time to load utils op: 0.001996278762817383 seconds 28: Time to load utils op: 0.002206087112426758 seconds 10: Time to load utils op: 0.0019860267639160156 seconds 13: Time to load utils op: 0.0017337799072265625 seconds 22: Time to load utils op: 0.002094268798828125 seconds 8: Time to load utils op: 0.0024802684783935547 seconds 8: Time to load utils op: 0.0023441314697265625 seconds 22: Time to load utils op: 0.002240896224975586 seconds 22: Time to load utils op: 0.002242565155029297 seconds 31: Time to load utils op: 0.002432584762573242 seconds 31: Time to load utils op: 0.0023064613342285156 seconds 31: Time to load utils op: 0.0024261474609375 seconds 31: Time to load utils op: 0.002350330352783203 seconds 31: Time to load utils op: 0.002266407012939453 seconds 31: Time to load utils op: 0.003100156784057617 seconds 4: Time to load utils op: 0.002814769744873047 seconds 8: Time to load utils op: 0.0027244091033935547 seconds 8: Time to load utils op: 0.0027010440826416016 seconds 4: Time to load utils op: 0.0027959346771240234 seconds 8: Time to load utils op: 0.002863168716430664 seconds 13: Time to load utils op: 0.002210855484008789 seconds 28: Time to load utils op: 0.0025560855865478516 seconds 28: Time to load utils op: 0.002641916275024414 seconds 22: Time to load utils op: 0.0027544498443603516 seconds 29: Time to load utils op: 0.002691507339477539 seconds 22: Time to load utils op: 0.0027785301208496094 seconds 29: Time to load utils op: 0.0026421546936035156 seconds 29: Time to load utils op: 0.0026721954345703125 seconds 22: Time to load utils op: 0.002797842025756836 seconds 29: Time to load utils op: 0.0027713775634765625 seconds 29: Time to load utils op: 0.002889871597290039 seconds 29: Time to load utils op: 0.0029273033142089844 seconds 4: Time to load utils op: 0.0033228397369384766 seconds 8: Time to load utils op: 0.003317117691040039 seconds 30: Time to load utils op: 0.0031244754791259766 seconds 4: Time to load utils op: 0.0033605098724365234 seconds 4: Time to load utils op: 0.0033416748046875 seconds 4: Time to load utils op: 0.0033223628997802734 seconds 27: Time to load utils op: 0.003070354461669922 seconds 9: Time to load utils op: 0.003236532211303711 seconds 10: Time to load utils op: 0.0031156539916992188 seconds 24: Time to load utils op: 0.0031502246856689453 seconds 10: Time to load utils op: 0.0032873153686523438 seconds 10: Time to load utils op: 0.003268003463745117 seconds 28: Time to load utils op: 0.003360748291015625 seconds 7: Time to load utils op: 0.0033359527587890625 seconds 22: Time to load utils op: 0.0034780502319335938 seconds 3: Time to load utils op: 0.0037908554077148438 seconds 10: Time to load utils op: 0.003376007080078125 seconds 29: Time to load utils op: 0.0035238265991210938 seconds 3: Time to load utils op: 0.003786802291870117 seconds 3: Time to load utils op: 0.003782033920288086 seconds 30: Time to load utils op: 0.0034914016723632812 seconds 28: Time to load utils op: 0.003522634506225586 seconds 3: Time to load utils op: 0.0038442611694335938 seconds 24: Time to load utils op: 0.003348827362060547 seconds 30: Time to load utils op: 0.0035588741302490234 seconds 3: Time to load utils op: 0.0038251876831054688 seconds 13: Time to load utils op: 0.0032074451446533203 seconds 7: Time to load utils op: 0.003447294235229492 seconds 7: Time to load utils op: 0.0035986900329589844 seconds 28: Time to load utils op: 0.003596782684326172 seconds 27: Time to load utils op: 0.0039234161376953125 seconds 30: Time to load utils op: 0.0037949085235595703 seconds 27: Time to load utils op: 0.003943681716918945 seconds 24: Time to load utils op: 0.003613710403442383 seconds 9: Time to load utils op: 0.0038640499114990234 seconds 24: Time to load utils op: 0.0036416053771972656 seconds 24: Time to load utils op: 0.003657817840576172 seconds 30: Time to load utils op: 0.0037500858306884766 seconds 30: Time to load utils op: 0.0037848949432373047 seconds 24: Time to load utils op: 0.0038034915924072266 secondsTime to load utils op: 0.0036127567291259766 seconds 24: 24: Time to load utils op: 0.003675222396850586 seconds 27: Time to load utils op: 0.00423431396484375 seconds 27: Time to load utils op: 0.004258394241333008 seconds 27: Time to load utils op: 0.004201650619506836 seconds 27: Time to load utils op: 0.004286289215087891 seconds 9: Time to load utils op: 0.004175424575805664 seconds 9: Time to load utils op: 0.004207134246826172 seconds 13: Time to load utils op: 0.004014730453491211 seconds 13: Time to load utils op: 0.0039119720458984375 seconds 13: Time to load utils op: 0.003836393356323242 seconds 13: Time to load utils op: 0.003851652145385742 seconds 9: Time to load utils op: 0.004218578338623047 seconds 9: Time to load utils op: 0.004167079925537109 seconds 9: Time to load utils op: 0.004188060760498047 seconds 9: Time to load utils op: 0.0042645931243896484 seconds 7: Time to load utils op: 0.004378080368041992 seconds 7: Time to load utils op: 0.004541873931884766 secondsTime to load utils op: 0.0044994354248046875 seconds 7: 7: Time to load utils op: 0.0043792724609375 seconds 7: Time to load utils op: 0.0044138431549072266 seconds 6: Time to load utils op: 0.0012576580047607422 seconds 6: Time to load utils op: 0.001973867416381836 seconds 6: Time to load utils op: 0.0019736289978027344 seconds 6: Time to load utils op: 0.002081632614135742 seconds 6: Time to load utils op: 0.0020263195037841797 secondsTime to load utils op: 0.00206756591796875 seconds 6: 6: Time to load utils op: 0.002079486846923828 seconds 6: Time to load utils op: 0.002062559127807617 seconds 20: Time to load utils op: 0.0009925365447998047 seconds 20: Time to load utils op: 0.001184225082397461 seconds 20: Time to load utils op: 0.0012273788452148438 seconds 20: Time to load utils op: 0.0011010169982910156 seconds 20: Time to load utils op: 0.0011816024780273438 seconds 20: Time to load utils op: 0.0011184215545654297 seconds 20: Time to load utils op: 0.0011701583862304688 seconds 20: Time to load utils op: 0.0011529922485351562 seconds 0: [2023-05-13 12:04:55,018] [INFO] [utils.py:827:see_memory_usage] end initialize_optimizer 0: [2023-05-13 12:04:55,019] [INFO] [utils.py:828:see_memory_usage] MA 15.82 GB Max_MA 15.82 GB CA 21.01 GB Max_CA 21 GB 0: [2023-05-13 12:04:55,019] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.33 GB, percent = 8.2% 26: Time to load utils op: 0.0011749267578125 seconds 25: Time to load utils op: 0.0010905265808105469 seconds 26: Time to load utils op: 0.002216339111328125 seconds 26: Time to load utils op: 0.002171039581298828 seconds 17: Time to load utils op: 0.0015790462493896484 seconds 26: Time to load utils op: 0.0020618438720703125 seconds 25: Time to load utils op: 0.0015180110931396484 seconds 26: Time to load utils op: 0.0022115707397460938 secondsTime to load utils op: 0.0022118091583251953 seconds 26: 26: Time to load utils op: 0.0022363662719726562 seconds 26: Time to load utils op: 0.0022363662719726562 seconds 25: Time to load utils op: 0.0014433860778808594 seconds 25: Time to load utils op: 0.0014677047729492188 seconds 25: Time to load utils op: 0.0015208721160888672 seconds 25: Time to load utils op: 0.0015463829040527344 seconds 25: Time to load utils op: 0.0015056133270263672 secondsTime to load utils op: 0.0014677047729492188 seconds 25: 17: Time to load utils op: 0.0016124248504638672 seconds 17: Time to load utils op: 0.0015783309936523438 seconds 17: Time to load utils op: 0.0016260147094726562 seconds 17: Time to load utils op: 0.0015189647674560547 secondsTime to load utils op: 0.0015430450439453125 seconds 17: 17: Time to load utils op: 0.0015859603881835938 seconds 17: Time to load utils op: 0.001659393310546875 seconds 0: [2023-05-13 12:04:55,125] [INFO] [utils.py:827:see_memory_usage] end bf16_optimizer 0: [2023-05-13 12:04:55,125] [INFO] [utils.py:828:see_memory_usage] MA 15.82 GB Max_MA 15.82 GB CA 21.01 GB Max_CA 21 GB 0: [2023-05-13 12:04:55,126] [INFO] [utils.py:836:see_memory_usage] CPU Virtual Memory: used = 41.33 GB, percent = 8.2% 0: [2023-05-13 12:04:55,126] [INFO] [logging.py:68:log_dist] [Rank 0] DeepSpeed Final Optimizer = FusedAdam 0: [2023-05-13 12:04:55,126] [INFO] [logging.py:68:log_dist] [Rank 0] DeepSpeed using client LR scheduler 0: [2023-05-13 12:04:55,126] [INFO] [logging.py:68:log_dist] [Rank 0] DeepSpeed LR Scheduler = 0: [2023-05-13 12:04:55,126] [INFO] [logging.py:68:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0002, 0.0002, 0.0002], mom=[(0.9, 0.999), (0.9, 0.999), (0.9, 0.999)] 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1007:print] DeepSpeedEngine configuration: 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] activation_checkpointing_config { 0: "partition_activations": false, 0: "contiguous_memory_optimization": false, 0: "cpu_checkpointing": false, 0: "number_checkpoints": null, 0: "synchronize_checkpoint_boundary": false, 0: "profile": false 0: } 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True} 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] amp_enabled .................. False 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] amp_params ................... False 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] autotuning_config ............ { 0: "enabled": false, 0: "start_step": null, 0: "end_step": null, 0: "metric_path": null, 0: "arg_mappings": null, 0: "metric": "throughput", 0: "model_info": null, 0: "results_dir": "/pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/autotuning_results", 0: "exps_dir": "/pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/autotuning_exps", 0: "overwrite": true, 0: "fast": true, 0: "start_profile_step": 3, 0: "end_profile_step": 5, 0: "tuner_type": "gridsearch", 0: "tuner_early_stopping": 5, 0: "tuner_num_trials": 50, 0: "model_info_path": null, 0: "mp_size": 1, 0: "max_train_batch_size": null, 0: "min_train_batch_size": 1, 0: "max_train_micro_batch_size_per_gpu": 1.024000e+03, 0: "min_train_micro_batch_size_per_gpu": 1, 0: "num_tuning_micro_batch_sizes": 3 0: } 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] bfloat16_enabled ............. True 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] checkpoint_parallel_write_pipeline False 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] checkpoint_tag_validation_enabled True 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] checkpoint_tag_validation_fail False 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] comms_config ................. 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] communication_data_type ...... None 0: [2023-05-13 12:04:55,127] [INFO] [config.py:1011:print] compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_pa 0: rameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}} 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] curriculum_enabled ........... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] curriculum_params ............ False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] dataloader_drop_last ......... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] disable_allgather ............ False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] dump_state ................... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] dynamic_loss_scale_args ...... None 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] eigenvalue_enabled ........... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] eigenvalue_gas_boundary_resolution 1 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] eigenvalue_layer_name ........ bert.encoder.layer 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] eigenvalue_layer_num ......... 0 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] eigenvalue_max_iter .......... 100 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] eigenvalue_stability ......... 1e-06 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] eigenvalue_tol ............... 0.01 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] eigenvalue_verbose ........... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] elasticity_enabled ........... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] flops_profiler_config ........ { 0: "enabled": false, 0: "profile_step": 1, 0: "module_depth": -1, 0: "top_modules": 1, 0: "detailed": true, 0: "output_file": null 0: } 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] fp16_auto_cast ............... None 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] fp16_enabled ................. False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] fp16_master_weights_and_gradients False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] global_rank .................. 0 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] gradient_accumulation_steps .. 1 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] gradient_clipping ............ 1.0 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] gradient_predivide_factor .... 1.0 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] initial_dynamic_scale ........ 1 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] load_universal_checkpoint .... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] loss_scale ................... 1.0 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] memory_breakdown ............. False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] monitor_config ............... 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] nebula_config ................ { 0: "enabled": false, 0: "persistent_storage_path": null, 0: "persistent_time_interval": 100, 0: "num_of_version_in_retention": 2, 0: "enable_nebula_load": true, 0: "load_path": null 0: } 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] optimizer_legacy_fusion ...... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] optimizer_name ............... None 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] optimizer_params ............. None 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0} 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] pld_enabled .................. False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] pld_params ................... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] prescale_gradients ........... False 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] scheduler_name ............... None 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] scheduler_params ............. None 0: [2023-05-13 12:04:55,128] [INFO] [config.py:1011:print] sparse_attention ............. None 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] sparse_gradients_enabled ..... False 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] steps_per_print .............. 2000 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] train_batch_size ............. 512 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] train_micro_batch_size_per_gpu 2 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] use_node_local_storage ....... False 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] wall_clock_breakdown ......... False 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] world_size ................... 256 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] zero_allow_untested_optimizer False 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] zero_config .................. stage=0 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500000000 allgather_partitions=True allgather_bucket_size=500000000 overlap_comm=False load_from_fp32_weights=True elastic_checkpoint=False offload_param=None offload_optimizer=None sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] zero_enabled ................. False 0: [2023-05-13 12:04:55,129] [INFO] [config.py:1011:print] zero_optimization_stage ...... 0 0: [2023-05-13 12:04:55,129] [INFO] [config.py:996:print_user_config] json = { 0: "train_micro_batch_size_per_gpu": 2, 0: "train_batch_size": 512, 0: "gradient_clipping": 1.0, 0: "zero_optimization": { 0: "stage": 0 0: }, 0: "bf16": { 0: "enabled": true 0: }, 0: "steps_per_print": 2.000000e+03, 0: "wall_clock_breakdown": false 0: } 0: Time to load utils op: 0.0004394054412841797 seconds 0: [2023-05-13 12:04:55,129] [INFO] [engine.py:87:__init__] CONFIG: micro_batches=1 micro_batch_size=2 0: [2023-05-13 12:04:55,192] [INFO] [engine.py:145:__init__] RANK=0 STAGE=0 LAYERS=41 [0, 41) STAGE_PARAMS=2809026560 (2809.027M) TOTAL_PARAMS=2809026560 (2809.027M) UNIQUE_PARAMS=2809026560 (2809.027M) 28: [2023-05-13 12:04:55,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 21: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 21: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 19: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 16: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 18: [2023-05-13 12:04:55,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 14: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 18: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 0: [2023-05-13 12:04:55,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:55,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 23: [2023-05-13 12:04:55,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 23: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:55,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 22: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 12: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 4: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 10: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 8: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 17: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 20: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 3: [2023-05-13 12:04:55,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 7: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 26: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 6: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 9: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 24: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 30: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 29: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 31: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 13: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 27: [2023-05-13 12:04:55,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 25: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt... 28: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 10: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 10: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 10: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 10: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 3: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 12: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 20: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 10: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 22: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 9: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 10: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/mp_rank_00_model_states.pt. 8: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 10: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:55,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:55,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:55,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:55,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:55,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:55,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:55,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:55,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:55,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:55,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:55,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:55,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:55,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:55,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:55,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:55,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:55,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:55,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:55,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:55,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:55,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:55,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:55,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:55,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:55,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:55,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:55,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:55,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:55,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:55,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:55,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:55,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:55,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:55,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:55,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:55,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:55,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:55,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:55,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:55,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:55,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:55,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,945] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:55,945] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:55,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:55,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:55,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:55,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:55,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:55,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:55,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:55,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:55,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:55,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:55,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:55,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:55,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:55,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:55,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:55,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:55,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:55,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:55,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:55,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:55,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:55,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:55,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:55,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:55,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:55,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:55,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:55,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:55,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:55,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:55,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:55,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:55,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:55,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:55,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:55,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:55,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:55,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:55,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:55,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:55,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:55,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:55,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:55,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:55,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:55,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:55,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:55,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:55,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:55,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:55,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:55,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:55,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:55,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:55,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:56,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 29: [2023-05-13 12:04:56,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:56,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:56,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:56,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:56,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:56,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:56,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:56,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:56,003] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,003] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:56,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:56,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:56,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:56,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:56,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:56,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:56,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:56,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:56,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:56,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:56,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 27: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 15: [2023-05-13 12:04:56,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:56,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:56,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:56,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:56,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:56,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:56,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:56,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:56,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:56,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:56,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 30: [2023-05-13 12:04:56,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:56,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:56,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:56,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:56,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:56,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:56,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:56,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:56,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:56,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:56,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 2: [2023-05-13 12:04:56,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:56,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:56,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:56,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:56,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:56,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:56,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:56,019] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:56,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:56,021] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:56,021] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:56,021] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:56,022] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:56,022] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:56,022] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:56,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:56,023] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:56,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:56,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:56,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:56,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:56,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:56,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:56,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 16: [2023-05-13 12:04:56,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 20: [2023-05-13 12:04:56,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:56,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:56,029] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:56,029] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:56,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:56,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 2: [2023-05-13 12:04:56,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:56,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:56,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:56,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:56,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:56,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:56,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:56,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:56,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:56,044] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:56,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:56,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 29: [2023-05-13 12:04:56,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,051] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:56,051] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:56,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:56,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 15: [2023-05-13 12:04:56,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,056] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:56,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:56,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:56,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:56,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 19: [2023-05-13 12:04:56,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:56,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:56,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:56,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:56,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:56,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:56,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:56,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 19: [2023-05-13 12:04:56,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 25: [2023-05-13 12:04:56,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:56,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:56,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:56,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 3: [2023-05-13 12:04:56,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 24: [2023-05-13 12:04:56,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:56,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:56,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:56,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:56,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:56,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:56,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:56,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 5: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:56,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 13: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 3: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 31: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,073] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:56,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,073] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,073] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 7: [2023-05-13 12:04:56,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:56,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:56,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:56,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:56,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:56,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 28: [2023-05-13 12:04:56,080] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:56,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 17: [2023-05-13 12:04:56,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 10: [2023-05-13 12:04:56,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 11: [2023-05-13 12:04:56,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 10: [2023-05-13 12:04:56,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:56,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:56,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:56,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:56,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 4: [2023-05-13 12:04:56,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:56,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 14: [2023-05-13 12:04:56,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 11: [2023-05-13 12:04:56,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 4: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 10: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 1: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:56,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:56,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 12: [2023-05-13 12:04:56,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:56,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:56,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:56,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:56,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 6: [2023-05-13 12:04:56,088] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,088] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:56,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 18: [2023-05-13 12:04:56,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 12: [2023-05-13 12:04:56,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 0: [2023-05-13 12:04:56,090] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 9: [2023-05-13 12:04:56,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 8: [2023-05-13 12:04:56,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,092] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:56,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,095] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,095] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,095] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,095] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:56,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:56,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:56,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:56,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:56,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:56,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:56,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 26: [2023-05-13 12:04:56,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:56,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:56,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 30: [2023-05-13 12:04:56,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,102] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 8: [2023-05-13 12:04:56,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,099] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:56,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:56,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:56,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:56,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 5: [2023-05-13 12:04:56,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,112] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,114] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,114] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:56,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:56,118] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:56,118] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:56,118] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:56,118] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,119] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,119] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,119] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,119] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,119] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:56,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,123] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 13: [2023-05-13 12:04:56,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:56,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:56,131] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:56,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,134] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:56,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:56,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,136] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:56,138] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 22: [2023-05-13 12:04:56,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:56,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 1: [2023-05-13 12:04:56,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,142] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,144] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,144] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,144] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,144] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:56,144] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 22: [2023-05-13 12:04:56,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 23: [2023-05-13 12:04:56,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:56,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:56,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 26: [2023-05-13 12:04:56,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt... 21: [2023-05-13 12:04:56,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,152] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,152] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,152] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,153] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 27: [2023-05-13 12:04:56,154] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,154] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 14: [2023-05-13 12:04:56,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 21: [2023-05-13 12:04:56,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 16: [2023-05-13 12:04:56,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 6: [2023-05-13 12:04:56,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,168] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,170] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,174] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,174] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 28: [2023-05-13 12:04:56,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 0: [2023-05-13 12:04:56,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,186] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,186] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,186] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 31: [2023-05-13 12:04:56,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 20: [2023-05-13 12:04:56,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 7: [2023-05-13 12:04:56,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 25: [2023-05-13 12:04:56,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 24: [2023-05-13 12:04:56,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 10: [2023-05-13 12:04:56,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,223] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,223] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 17: [2023-05-13 12:04:56,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 18: [2023-05-13 12:04:56,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 23: [2023-05-13 12:04:56,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_01-model_00-model_states.pt. 9: [2023-05-13 12:04:56,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:56,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:56,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:56,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 12: [2023-05-13 12:04:56,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:56,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:56,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:56,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:56,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 12: [2023-05-13 12:04:56,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:56,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:56,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:56,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:56,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:56,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 15: [2023-05-13 12:04:56,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:56,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 29: [2023-05-13 12:04:56,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 15: [2023-05-13 12:04:56,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 6: [2023-05-13 12:04:56,648] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:56,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:56,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:56,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:56,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:56,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:56,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:56,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:56,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:56,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 29: [2023-05-13 12:04:56,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:56,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 8: [2023-05-13 12:04:56,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:56,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 2: [2023-05-13 12:04:56,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:56,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:56,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:56,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 6: [2023-05-13 12:04:56,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:56,692] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,692] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 13: [2023-05-13 12:04:56,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 11: [2023-05-13 12:04:56,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:56,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:56,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 5: [2023-05-13 12:04:56,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:56,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:56,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:56,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 19: [2023-05-13 12:04:56,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:56,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:56,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 11: [2023-05-13 12:04:56,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:56,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:56,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:56,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:56,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:56,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 2: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 3: [2023-05-13 12:04:56,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 4: [2023-05-13 12:04:56,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:56,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:56,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:56,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:56,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 28: [2023-05-13 12:04:56,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:56,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:56,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:56,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:56,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 4: [2023-05-13 12:04:56,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:56,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:56,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:56,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 3: [2023-05-13 12:04:56,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:56,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:56,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:56,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:56,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:56,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:56,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,728] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:56,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:56,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:56,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:56,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:56,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:56,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:56,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:56,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 19: [2023-05-13 12:04:56,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 13: [2023-05-13 12:04:56,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:56,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:56,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:56,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:56,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:56,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:56,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 5: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 28: [2023-05-13 12:04:56,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:56,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:56,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:56,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 1: [2023-05-13 12:04:56,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:56,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:56,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 1: [2023-05-13 12:04:56,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 0: [2023-05-13 12:04:56,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 30: [2023-05-13 12:04:56,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:56,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 27: [2023-05-13 12:04:56,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 8: [2023-05-13 12:04:56,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:56,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 26: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 20: [2023-05-13 12:04:56,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 17: [2023-05-13 12:04:56,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 30: [2023-05-13 12:04:56,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:56,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 27: [2023-05-13 12:04:56,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:56,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:56,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:56,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:56,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:56,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:56,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:56,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:56,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:56,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 22: [2023-05-13 12:04:56,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:56,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:56,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 22: [2023-05-13 12:04:56,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 0: [2023-05-13 12:04:56,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:56,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:56,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,822] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 31: [2023-05-13 12:04:56,822] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 16: [2023-05-13 12:04:56,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:56,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 24: [2023-05-13 12:04:56,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 9: [2023-05-13 12:04:56,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 16: [2023-05-13 12:04:56,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:56,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:56,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:56,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:56,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:56,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 7: [2023-05-13 12:04:56,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 31: [2023-05-13 12:04:56,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:56,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 23: [2023-05-13 12:04:56,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 10: [2023-05-13 12:04:56,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:56,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 24: [2023-05-13 12:04:56,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,850] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,850] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,850] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 10: [2023-05-13 12:04:56,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:56,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:56,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:56,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 18: [2023-05-13 12:04:56,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:56,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:56,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:56,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:56,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 7: [2023-05-13 12:04:56,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:56,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:56,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:56,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:56,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:56,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:56,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 26: [2023-05-13 12:04:56,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 21: [2023-05-13 12:04:56,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 25: [2023-05-13 12:04:56,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 14: [2023-05-13 12:04:56,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt... 18: [2023-05-13 12:04:56,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:56,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 20: [2023-05-13 12:04:56,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:56,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:56,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:56,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:56,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:56,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:56,893] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,893] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:56,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 17: [2023-05-13 12:04:56,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:56,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:56,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:56,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:56,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:56,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:56,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:56,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:56,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:56,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:56,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:56,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:56,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:56,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:56,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:56,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:56,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:56,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:56,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:56,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 25: [2023-05-13 12:04:56,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:56,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 23: [2023-05-13 12:04:56,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:56,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:56,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:56,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:56,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 14: [2023-05-13 12:04:56,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 21: [2023-05-13 12:04:56,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_03-model_00-model_states.pt. 9: [2023-05-13 12:04:56,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:56,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:56,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:56,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:56,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:56,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:56,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:56,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:56,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,117] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:57,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:57,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:57,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,134] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:57,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:57,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:57,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 5: [2023-05-13 12:04:57,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:57,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:57,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:57,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:57,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:57,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:57,142] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:57,142] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 11: [2023-05-13 12:04:57,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 15: [2023-05-13 12:04:57,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 11: [2023-05-13 12:04:57,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,217] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,217] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 6: [2023-05-13 12:04:57,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,250] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 27: [2023-05-13 12:04:57,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:57,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:57,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:57,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:57,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:57,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:57,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 28: [2023-05-13 12:04:57,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,299] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,299] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,299] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 12: [2023-05-13 12:04:57,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:57,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:57,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:57,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:57,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:57,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:57,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:57,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:57,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:57,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:57,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:57,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:57,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 27: [2023-05-13 12:04:57,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:57,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:57,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:57,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:57,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:57,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:57,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:57,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:57,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:57,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 4: [2023-05-13 12:04:57,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 12: [2023-05-13 12:04:57,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,319] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,319] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,320] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,320] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,320] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,325] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,325] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,325] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:57,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:57,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:57,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:57,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,335] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,335] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,335] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:57,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 10: [2023-05-13 12:04:57,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:57,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:57,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:57,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 3: [2023-05-13 12:04:57,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:57,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:57,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:57,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:57,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:57,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:57,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:57,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:57,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:57,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:57,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:57,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:57,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:57,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:57,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:57,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 16: [2023-05-13 12:04:57,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,362] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:57,362] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:57,362] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:57,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:57,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:57,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:57,364] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,369] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,369] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,369] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 16: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 31: [2023-05-13 12:04:57,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,372] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,372] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,372] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 31: [2023-05-13 12:04:57,374] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 28: [2023-05-13 12:04:57,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:57,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:57,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 13: [2023-05-13 12:04:57,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,381] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,381] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,382] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:57,382] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:57,382] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 29: [2023-05-13 12:04:57,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,393] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,393] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,393] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 29: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 30: [2023-05-13 12:04:57,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,399] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,399] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 22: [2023-05-13 12:04:57,399] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 22: [2023-05-13 12:04:57,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 1: [2023-05-13 12:04:57,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 4: [2023-05-13 12:04:57,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 8: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:57,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:57,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 9: [2023-05-13 12:04:57,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:57,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 19: [2023-05-13 12:04:57,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 14: [2023-05-13 12:04:57,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,417] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 13: [2023-05-13 12:04:57,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:57,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:57,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:57,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 8: [2023-05-13 12:04:57,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:57,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:57,426] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 30: [2023-05-13 12:04:57,426] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,426] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 1: [2023-05-13 12:04:57,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 26: [2023-05-13 12:04:57,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,434] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 3: [2023-05-13 12:04:57,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 10: [2023-05-13 12:04:57,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 26: [2023-05-13 12:04:57,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:57,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 14: [2023-05-13 12:04:57,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:57,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:57,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:57,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:57,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:57,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:57,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:57,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:57,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:57,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 17: [2023-05-13 12:04:57,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:57,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:57,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:57,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:57,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 19: [2023-05-13 12:04:57,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,476] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:57,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:57,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:57,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:57,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 7: [2023-05-13 12:04:57,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 7: [2023-05-13 12:04:57,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:57,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:57,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:57,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:57,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:57,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:57,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 21: [2023-05-13 12:04:57,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 21: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,520] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 9: [2023-05-13 12:04:57,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:57,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:57,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:57,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:57,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:57,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:57,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 24: [2023-05-13 12:04:57,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:57,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 0: [2023-05-13 12:04:57,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:57,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,536] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,536] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,536] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,536] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,536] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:57,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:57,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:57,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 2: [2023-05-13 12:04:57,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 25: [2023-05-13 12:04:57,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 20: [2023-05-13 12:04:57,543] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 15: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 15: [2023-05-13 12:04:57,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:57,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:57,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:57,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:57,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:57,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:57,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:57,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 23: [2023-05-13 12:04:57,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 18: [2023-05-13 12:04:57,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt... 0: [2023-05-13 12:04:57,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:57,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 17: [2023-05-13 12:04:57,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 23: [2023-05-13 12:04:57,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 23: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 23: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 23: [2023-05-13 12:04:57,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 15: [2023-05-13 12:04:57,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:57,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:57,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:57,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:57,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 23: [2023-05-13 12:04:57,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 23: [2023-05-13 12:04:57,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 24: [2023-05-13 12:04:57,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 6: [2023-05-13 12:04:57,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 6: [2023-05-13 12:04:57,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 6: [2023-05-13 12:04:57,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 2: [2023-05-13 12:04:57,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 25: [2023-05-13 12:04:57,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 5: [2023-05-13 12:04:57,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 2: [2023-05-13 12:04:57,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:57,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 18: [2023-05-13 12:04:57,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_04-model_00-model_states.pt. 20: [2023-05-13 12:04:57,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 6: [2023-05-13 12:04:57,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 6: [2023-05-13 12:04:57,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:57,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:57,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:57,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:57,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:57,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:57,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 6: [2023-05-13 12:04:57,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:57,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:57,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:57,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 2: [2023-05-13 12:04:57,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:57,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:57,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:57,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:57,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:57,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:57,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:57,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,658] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,658] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,664] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 12: [2023-05-13 12:04:57,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:57,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:57,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:57,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 5: [2023-05-13 12:04:57,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:57,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:57,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 5: [2023-05-13 12:04:57,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:57,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:57,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:57,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:57,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,720] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,720] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:57,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:57,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:57,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:57,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:57,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:57,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:57,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 12: [2023-05-13 12:04:57,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:57,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:57,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:57,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 4: [2023-05-13 12:04:57,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:57,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:57,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:57,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 4: [2023-05-13 12:04:57,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:57,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:57,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:57,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 7: [2023-05-13 12:04:57,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:57,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:57,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:57,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:57,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:57,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:57,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 28: [2023-05-13 12:04:57,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:57,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:57,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 8: [2023-05-13 12:04:57,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:57,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 11: [2023-05-13 12:04:57,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 28: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 16: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 16: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,872] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:57,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,872] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:57,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:57,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 30: [2023-05-13 12:04:57,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 8: [2023-05-13 12:04:57,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:57,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:57,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 1: [2023-05-13 12:04:57,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 31: [2023-05-13 12:04:57,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:57,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:57,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:57,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:57,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 31: [2023-05-13 12:04:57,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:57,888] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:57,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:57,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:57,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:57,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:57,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 30: [2023-05-13 12:04:57,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 30: [2023-05-13 12:04:57,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 30: [2023-05-13 12:04:57,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:57,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 30: [2023-05-13 12:04:57,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 28: [2023-05-13 12:04:57,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:57,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:57,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:57,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 30: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:57,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 28: [2023-05-13 12:04:57,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:57,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:57,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 11: [2023-05-13 12:04:57,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:57,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:57,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 28: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 28: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 10: [2023-05-13 12:04:57,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 22: [2023-05-13 12:04:57,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:57,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:57,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 14: [2023-05-13 12:04:57,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:57,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:57,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:57,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:57,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:57,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:57,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:57,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 7: [2023-05-13 12:04:57,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:57,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:57,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:57,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:57,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 21: [2023-05-13 12:04:57,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:57,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:57,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,949] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,949] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 20: [2023-05-13 12:04:57,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:57,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:57,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 29: [2023-05-13 12:04:57,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:57,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:57,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:57,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 3: [2023-05-13 12:04:57,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 29: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:57,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:57,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 21: [2023-05-13 12:04:57,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:57,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:57,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:57,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:57,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:57,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:57,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:57,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:57,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:57,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:57,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:57,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:57,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 3: [2023-05-13 12:04:57,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:57,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:57,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:57,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:57,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 26: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:57,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:57,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 25: [2023-05-13 12:04:57,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:57,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 9: [2023-05-13 12:04:57,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 9: [2023-05-13 12:04:57,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:57,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 26: [2023-05-13 12:04:57,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:57,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:57,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:57,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:57,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:57,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:57,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:57,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:57,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:57,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:57,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:57,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 13: [2023-05-13 12:04:57,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:57,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 1: [2023-05-13 12:04:57,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:57,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:57,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:57,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:57,989] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,989] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 14: [2023-05-13 12:04:57,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:57,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:57,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:57,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 17: [2023-05-13 12:04:57,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 10: [2023-05-13 12:04:57,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:57,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:57,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 22: [2023-05-13 12:04:57,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:57,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:57,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:57,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:57,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:57,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 27: [2023-05-13 12:04:57,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:57,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:57,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:57,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:57,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 27: [2023-05-13 12:04:57,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:57,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 19: [2023-05-13 12:04:57,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:57,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 20: [2023-05-13 12:04:57,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:57,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:57,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,013] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,013] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:58,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 13: [2023-05-13 12:04:58,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,019] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 24: [2023-05-13 12:04:58,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 23: [2023-05-13 12:04:58,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:58,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 23: [2023-05-13 12:04:58,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 23: [2023-05-13 12:04:58,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:58,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 24: [2023-05-13 12:04:58,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 23: [2023-05-13 12:04:58,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:58,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:58,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:58,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 17: [2023-05-13 12:04:58,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:58,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:58,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:58,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 18: [2023-05-13 12:04:58,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt... 0: [2023-05-13 12:04:58,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:58,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:58,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:58,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:58,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:58,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:58,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:58,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:58,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:58,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:58,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:58,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:58,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:58,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:58,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:58,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 25: [2023-05-13 12:04:58,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:58,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:58,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:58,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:58,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:58,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:58,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 0: [2023-05-13 12:04:58,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 23: [2023-05-13 12:04:58,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:58,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 18: [2023-05-13 12:04:58,094] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_05-model_00-model_states.pt. 19: [2023-05-13 12:04:58,096] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,098] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,099] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,099] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,112] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,112] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,120] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,121] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,121] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 6: [2023-05-13 12:04:58,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 6: [2023-05-13 12:04:58,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 6: [2023-05-13 12:04:58,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 6: [2023-05-13 12:04:58,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 6: [2023-05-13 12:04:58,360] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,569] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:58,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:58,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:58,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:58,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:58,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,663] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,664] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:58,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:58,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:58,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 31: [2023-05-13 12:04:58,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:58,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:58,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:58,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:58,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:58,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:58,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:58,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 31: [2023-05-13 12:04:58,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,723] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,728] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:58,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:58,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:58,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:58,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:58,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:58,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:58,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:58,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:58,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:58,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:58,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:58,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:58,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:58,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:58,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 21: [2023-05-13 12:04:58,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:58,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:58,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:58,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:58,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 8: [2023-05-13 12:04:58,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:58,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 15: [2023-05-13 12:04:58,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:58,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 8: [2023-05-13 12:04:58,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:58,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:58,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:58,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:58,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:58,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:58,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:58,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,837] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:58,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:58,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:58,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:58,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:58,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:58,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:58,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:58,844] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,844] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 15: [2023-05-13 12:04:58,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:58,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:58,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:58,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:58,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:58,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:58,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:58,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:58,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:58,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 2: [2023-05-13 12:04:58,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:58,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:58,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:58,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 18: [2023-05-13 12:04:58,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 6: [2023-05-13 12:04:58,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:58,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:58,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:58,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:58,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:58,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,876] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,876] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 21: [2023-05-13 12:04:58,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:58,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:58,876] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 6: [2023-05-13 12:04:58,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:58,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:58,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:58,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:58,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:58,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:58,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:58,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:58,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:58,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:58,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:58,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:58,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 20: [2023-05-13 12:04:58,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 2: [2023-05-13 12:04:58,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:58,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:58,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:58,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:58,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:58,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:58,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:58,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:58,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:58,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:58,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:58,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:58,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 24: [2023-05-13 12:04:58,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:58,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,888] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,888] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:58,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 24: [2023-05-13 12:04:58,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:58,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:58,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:58,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:58,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:58,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:58,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 11: [2023-05-13 12:04:58,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:58,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:58,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:58,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 11: [2023-05-13 12:04:58,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:58,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:58,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:58,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:58,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:58,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:58,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 18: [2023-05-13 12:04:58,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:58,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:58,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:58,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:58,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:58,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:58,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:58,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:58,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:58,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:58,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:58,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:58,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:58,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:58,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:58,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:58,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:58,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:58,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:58,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:58,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:58,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:58,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:58,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:58,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:58,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:58,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:58,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:58,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:58,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 25: [2023-05-13 12:04:58,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 1: [2023-05-13 12:04:58,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:58,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:58,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:58,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 3: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 28: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:58,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:58,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 4: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 23: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 23: [2023-05-13 12:04:58,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 30: [2023-05-13 12:04:58,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 30: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 30: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 30: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 30: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 27: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 7: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 20: [2023-05-13 12:04:58,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:58,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:58,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:58,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 10: [2023-05-13 12:04:58,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:58,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:58,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:58,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:58,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:58,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:58,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:58,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:58,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 28: [2023-05-13 12:04:58,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:58,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 22: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 28: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 28: [2023-05-13 12:04:58,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:58,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:58,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 4: [2023-05-13 12:04:58,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:58,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:58,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:58,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 9: [2023-05-13 12:04:58,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:58,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 5: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:58,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 0: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 17: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 3: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:58,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:58,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 13: [2023-05-13 12:04:58,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:58,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:58,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:58,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:58,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:58,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:58,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 16: [2023-05-13 12:04:58,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:58,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:58,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 29: [2023-05-13 12:04:58,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 7: [2023-05-13 12:04:58,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:58,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:58,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:58,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:58,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:58,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:58,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:58,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 26: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 12: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:58,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:58,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 25: [2023-05-13 12:04:58,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:58,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:58,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:58,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 19: [2023-05-13 12:04:58,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 14: [2023-05-13 12:04:58,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:58,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt... 10: [2023-05-13 12:04:58,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:58,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:58,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:58,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:58,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:58,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:58,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:58,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:58,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 16: [2023-05-13 12:04:58,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:58,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:58,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:58,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:58,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:58,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:59,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:59,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:59,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:59,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:59,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:59,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,008] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:59,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,013] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:59,013] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:59,013] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:59,013] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 5: [2023-05-13 12:04:59,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 12: [2023-05-13 12:04:59,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:59,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,021] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,021] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:59,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 23: [2023-05-13 12:04:59,023] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,023] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,023] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:59,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:59,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:59,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:59,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:59,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:59,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 16: [2023-05-13 12:04:59,028] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,028] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:59,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,029] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:59,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:59,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:59,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:59,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:59,033] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:59,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:59,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:59,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:59,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:59,038] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:59,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:59,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:59,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:59,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:59,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:59,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:59,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:59,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:59,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,044] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:59,044] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:59,044] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 29: [2023-05-13 12:04:59,044] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 14: [2023-05-13 12:04:59,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 23: [2023-05-13 12:04:59,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 1: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 26: [2023-05-13 12:04:59,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:59,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:59,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:59,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 23: [2023-05-13 12:04:59,052] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 23: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 23: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 9: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:59,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 23: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 13: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 23: [2023-05-13 12:04:59,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:59,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:59,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:59,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:59,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 28: [2023-05-13 12:04:59,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:59,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:59,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:59,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:59,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 27: [2023-05-13 12:04:59,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 30: [2023-05-13 12:04:59,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:59,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:59,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:59,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 0: [2023-05-13 12:04:59,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 19: [2023-05-13 12:04:59,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:59,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:59,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:59,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 17: [2023-05-13 12:04:59,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_06-model_00-model_states.pt. 22: [2023-05-13 12:04:59,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,096] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,096] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,096] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,098] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,098] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,099] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:59,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:59,117] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:59,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:59,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,131] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,131] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:59,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:59,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:59,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:59,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:04:59,191] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,198] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,198] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:59,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:59,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:59,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:59,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:59,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 2: [2023-05-13 12:04:59,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 15: [2023-05-13 12:04:59,223] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:04:59,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:59,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:59,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:59,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:59,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:59,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:59,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:59,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:59,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 18: [2023-05-13 12:04:59,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:59,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:59,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:59,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 15: [2023-05-13 12:04:59,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:04:59,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:59,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:59,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:59,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:59,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:59,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:59,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:59,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:59,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:59,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:59,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:59,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:59,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:59,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 11: [2023-05-13 12:04:59,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:59,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,268] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,268] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,268] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,268] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,268] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,268] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,269] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,269] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:59,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:59,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:59,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:59,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:59,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:59,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,282] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,282] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,282] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,297] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:59,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:59,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:59,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:59,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:59,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 6: [2023-05-13 12:04:59,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:59,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:59,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:59,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:59,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:59,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 6: [2023-05-13 12:04:59,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 2: [2023-05-13 12:04:59,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,326] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:59,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 24: [2023-05-13 12:04:59,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:59,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,338] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,338] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,338] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:59,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 20: [2023-05-13 12:04:59,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:59,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:59,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,360] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 12: [2023-05-13 12:04:59,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,364] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,364] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,364] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:59,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 3: [2023-05-13 12:04:59,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,372] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 16: [2023-05-13 12:04:59,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:59,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 23: [2023-05-13 12:04:59,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 23: [2023-05-13 12:04:59,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:59,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:59,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 12: [2023-05-13 12:04:59,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 5: [2023-05-13 12:04:59,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 5: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 9: [2023-05-13 12:04:59,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:59,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:59,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:59,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 25: [2023-05-13 12:04:59,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:04:59,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:59,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:59,398] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,398] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 25: [2023-05-13 12:04:59,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 21: [2023-05-13 12:04:59,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,420] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,420] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,420] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,420] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 26: [2023-05-13 12:04:59,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 1: [2023-05-13 12:04:59,426] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 31: [2023-05-13 12:04:59,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 27: [2023-05-13 12:04:59,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 19: [2023-05-13 12:04:59,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 4: [2023-05-13 12:04:59,434] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,434] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 28: [2023-05-13 12:04:59,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 8: [2023-05-13 12:04:59,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,437] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 14: [2023-05-13 12:04:59,443] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,443] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,443] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,443] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,443] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,443] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 14: [2023-05-13 12:04:59,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 23: [2023-05-13 12:04:59,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:04:59,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 0: [2023-05-13 12:04:59,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 8: [2023-05-13 12:04:59,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:59,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 18: [2023-05-13 12:04:59,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 28: [2023-05-13 12:04:59,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 4: [2023-05-13 12:04:59,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,476] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,476] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,479] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:04:59,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 10: [2023-05-13 12:04:59,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 21: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 26: [2023-05-13 12:04:59,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 30: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 13: [2023-05-13 12:04:59,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:59,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 30: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 27: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 29: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 7: [2023-05-13 12:04:59,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 22: [2023-05-13 12:04:59,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt... 17: [2023-05-13 12:04:59,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 10: [2023-05-13 12:04:59,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 18: [2023-05-13 12:04:59,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:04:59,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:04:59,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 20: [2023-05-13 12:04:59,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 31: [2023-05-13 12:04:59,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 9: [2023-05-13 12:04:59,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 17: [2023-05-13 12:04:59,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:04:59,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:04:59,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,517] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,517] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,517] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,517] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,517] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 11: [2023-05-13 12:04:59,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 24: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 19: [2023-05-13 12:04:59,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 18: [2023-05-13 12:04:59,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:04:59,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:04:59,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:04:59,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:04:59,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:04:59,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 3: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 29: [2023-05-13 12:04:59,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 1: [2023-05-13 12:04:59,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:04:59,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 22: [2023-05-13 12:04:59,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 0: [2023-05-13 12:04:59,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,561] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,561] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,561] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 13: [2023-05-13 12:04:59,563] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_07-model_00-model_states.pt. 7: [2023-05-13 12:04:59,563] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,563] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:04:59,576] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,576] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,576] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 11: [2023-05-13 12:04:59,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,583] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,583] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,583] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 20: [2023-05-13 12:04:59,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 20: [2023-05-13 12:04:59,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,625] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,625] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 3: [2023-05-13 12:04:59,625] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 24: [2023-05-13 12:04:59,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:04:59,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 3: [2023-05-13 12:04:59,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:04:59,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:04:59,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:04:59,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 4: [2023-05-13 12:04:59,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:04:59,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:04:59,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,687] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:04:59,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:04:59,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 14: [2023-05-13 12:04:59,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:04:59,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:04:59,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,714] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 2: [2023-05-13 12:04:59,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:04:59,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:04:59,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:04:59,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:04:59,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 2: [2023-05-13 12:04:59,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:04:59,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 25: [2023-05-13 12:04:59,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 25: [2023-05-13 12:04:59,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:04:59,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:04:59,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:04:59,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:04:59,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:04:59,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:04:59,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:04:59,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:04:59,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:04:59,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 4: [2023-05-13 12:04:59,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 14: [2023-05-13 12:04:59,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:04:59,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 28: [2023-05-13 12:04:59,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 12: [2023-05-13 12:04:59,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:04:59,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:04:59,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:04:59,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 16: [2023-05-13 12:04:59,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:04:59,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 12: [2023-05-13 12:04:59,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:04:59,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:04:59,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 28: [2023-05-13 12:04:59,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:04:59,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:04:59,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:04:59,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:04:59,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:04:59,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:04:59,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:04:59,748] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:04:59,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:04:59,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:04:59,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:04:59,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:04:59,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 23: [2023-05-13 12:04:59,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:04:59,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:04:59,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:04:59,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:04:59,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:04:59,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:04:59,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:04:59,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:04:59,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 16: [2023-05-13 12:04:59,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:04:59,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:04:59,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 9: [2023-05-13 12:04:59,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:04:59,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 15: [2023-05-13 12:04:59,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:04:59,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,780] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:04:59,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:04:59,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:04:59,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:04:59,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:04:59,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:04:59,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 10: [2023-05-13 12:04:59,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:04:59,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 5: [2023-05-13 12:04:59,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:04:59,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 15: [2023-05-13 12:04:59,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:04:59,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:04:59,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 10: [2023-05-13 12:04:59,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:04:59,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:04:59,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:04:59,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:04:59,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 30: [2023-05-13 12:04:59,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 5: [2023-05-13 12:04:59,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 19: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:04:59,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 15: [2023-05-13 12:04:59,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:04:59,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:04:59,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:04:59,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:04:59,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:04:59,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:04:59,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 26: [2023-05-13 12:04:59,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 23: [2023-05-13 12:04:59,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:04:59,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:04:59,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 6: [2023-05-13 12:04:59,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:04:59,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 9: [2023-05-13 12:04:59,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:04:59,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 29: [2023-05-13 12:04:59,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:04:59,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:04:59,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 1: [2023-05-13 12:04:59,820] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,820] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,820] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:04:59,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 21: [2023-05-13 12:04:59,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:04:59,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:04:59,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:04:59,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:04:59,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 27: [2023-05-13 12:04:59,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 19: [2023-05-13 12:04:59,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:04:59,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:04:59,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:04:59,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:04:59,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:04:59,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:04:59,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:04:59,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 17: [2023-05-13 12:04:59,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 30: [2023-05-13 12:04:59,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:04:59,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 31: [2023-05-13 12:04:59,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:04:59,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:04:59,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 1: [2023-05-13 12:04:59,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,835] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,835] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,835] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 1: [2023-05-13 12:04:59,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:04:59,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:04:59,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 13: [2023-05-13 12:04:59,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:04:59,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:04:59,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:04:59,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 8: [2023-05-13 12:04:59,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:04:59,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 0: [2023-05-13 12:04:59,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 22: [2023-05-13 12:04:59,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt... 7: [2023-05-13 12:04:59,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:04:59,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:04:59,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:04:59,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:04:59,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 8: [2023-05-13 12:04:59,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:04:59,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:04:59,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:04:59,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:04:59,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:04:59,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 26: [2023-05-13 12:04:59,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 13: [2023-05-13 12:04:59,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 22: [2023-05-13 12:04:59,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:04:59,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:04:59,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:04:59,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:04:59,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:04:59,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:04:59,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:04:59,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:04:59,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:04:59,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:04:59,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:04:59,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:04:59,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:04:59,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:04:59,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:04:59,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:04:59,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:04:59,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:04:59,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:04:59,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:04:59,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:04:59,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:04:59,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:04:59,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:04:59,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:04:59,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:04:59,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 1: [2023-05-13 12:04:59,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 31: [2023-05-13 12:04:59,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:04:59,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:04:59,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:04:59,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:04:59,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 29: [2023-05-13 12:04:59,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:04:59,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 21: [2023-05-13 12:04:59,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:04:59,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:04:59,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:04:59,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:04:59,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:04:59,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:04:59,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:04:59,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:04:59,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:04:59,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:04:59,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:04:59,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:04:59,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:04:59,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:04:59,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:04:59,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:04:59,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:04:59,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:04:59,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:04:59,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:04:59,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 17: [2023-05-13 12:04:59,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 27: [2023-05-13 12:04:59,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:04:59,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:04:59,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:04:59,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 6: [2023-05-13 12:04:59,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:04:59,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:04:59,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:04:59,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:04:59,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:04:59,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 11: [2023-05-13 12:04:59,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:04:59,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 0: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 7: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:04:59,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:04:59,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 1: [2023-05-13 12:04:59,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_08-model_00-model_states.pt. 24: [2023-05-13 12:04:59,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:04:59,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:04:59,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:04:59,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:04:59,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:04:59,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:04:59,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:04:59,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 1: [2023-05-13 12:04:59,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 1: [2023-05-13 12:04:59,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 24: [2023-05-13 12:04:59,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:04:59,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:04:59,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:04:59,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:04:59,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:04:59,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:04:59,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:04:59,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:04:59,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:04:59,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:04:59,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:04:59,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:04:59,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:04:59,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:04:59,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:04:59,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:04:59,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:04:59,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:04:59,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:04:59,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:04:59,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:04:59,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:04:59,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:04:59,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:04:59,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:04:59,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:04:59,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:04:59,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:04:59,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:04:59,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:04:59,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:04:59,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:04:59,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 1: [2023-05-13 12:04:59,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 1: [2023-05-13 12:04:59,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 20: [2023-05-13 12:04:59,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:04:59,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:04:59,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:04:59,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:05:00,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:05:00,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:05:00,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:05:00,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:05:00,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:05:00,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:05:00,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:05:00,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,022] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:05:00,022] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:05:00,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:05:00,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:05:00,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:05:00,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:05:00,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:05:00,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 4: [2023-05-13 12:05:00,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 11: [2023-05-13 12:05:00,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,028] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,028] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,028] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 24: [2023-05-13 12:05:00,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,038] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:05:00,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:05:00,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:05:00,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:05:00,040] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:05:00,040] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:05:00,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 23: [2023-05-13 12:05:00,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:05:00,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 3: [2023-05-13 12:05:00,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:05:00,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 3: [2023-05-13 12:05:00,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 25: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 12: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 4: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:05:00,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:05:00,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:05:00,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:05:00,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:05:00,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:05:00,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 18: [2023-05-13 12:05:00,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:05:00,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:05:00,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,052] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:05:00,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:05:00,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:05:00,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:05:00,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:05:00,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:05:00,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:05:00,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:05:00,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:05:00,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:05:00,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:05:00,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:05:00,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:05:00,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:05:00,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:05:00,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:05:00,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:05:00,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 5: [2023-05-13 12:05:00,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:05:00,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:05:00,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:05:00,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:05:00,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:05:00,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:05:00,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:05:00,114] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:05:00,114] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:05:00,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:05:00,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:05:00,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:05:00,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:05:00,117] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:05:00,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 18: [2023-05-13 12:05:00,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 12: [2023-05-13 12:05:00,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 23: [2023-05-13 12:05:00,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 25: [2023-05-13 12:05:00,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:05:00,142] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,142] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 12: [2023-05-13 12:05:00,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,145] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,145] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 10: [2023-05-13 12:05:00,153] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:05:00,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:05:00,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:05:00,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:05:00,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:05:00,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:05:00,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:05:00,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 31: [2023-05-13 12:05:00,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:05:00,163] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:05:00,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:05:00,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:05:00,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:05:00,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:05:00,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:05:00,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:05:00,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:05:00,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:05:00,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 22: [2023-05-13 12:05:00,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 10: [2023-05-13 12:05:00,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:05:00,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:05:00,175] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:05:00,175] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:05:00,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:05:00,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:05:00,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:05:00,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:05:00,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 5: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:05:00,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 9: [2023-05-13 12:05:00,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:05:00,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:05:00,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:05:00,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:05:00,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,186] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,186] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:05:00,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:05:00,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:05:00,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 9: [2023-05-13 12:05:00,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:05:00,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 16: [2023-05-13 12:05:00,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:05:00,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:05:00,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:05:00,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:05:00,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:05:00,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:05:00,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:05:00,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:05:00,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:05:00,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:05:00,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:05:00,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:05:00,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 19: [2023-05-13 12:05:00,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:05:00,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:05:00,207] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,209] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,209] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:05:00,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:05:00,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:05:00,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 30: [2023-05-13 12:05:00,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:05:00,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:05:00,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 16: [2023-05-13 12:05:00,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:05:00,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:05:00,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:05:00,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:05:00,217] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:05:00,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:05:00,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:05:00,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:05:00,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:05:00,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 29: [2023-05-13 12:05:00,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:05:00,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 6: [2023-05-13 12:05:00,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:05:00,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,225] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,225] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,225] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:05:00,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:05:00,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:05:00,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 1: [2023-05-13 12:05:00,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 1: [2023-05-13 12:05:00,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:05:00,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:05:00,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 15: [2023-05-13 12:05:00,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:05:00,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 30: [2023-05-13 12:05:00,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 7: [2023-05-13 12:05:00,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:05:00,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 31: [2023-05-13 12:05:00,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:05:00,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 22: [2023-05-13 12:05:00,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:05:00,254] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:05:00,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,256] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:05:00,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:05:00,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:05:00,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:05:00,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:05:00,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:05:00,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 27: [2023-05-13 12:05:00,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 28: [2023-05-13 12:05:00,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 28: [2023-05-13 12:05:00,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 21: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 13: [2023-05-13 12:05:00,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:05:00,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,269] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:05:00,269] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 26: [2023-05-13 12:05:00,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:05:00,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:05:00,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:05:00,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 19: [2023-05-13 12:05:00,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 21: [2023-05-13 12:05:00,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 0: [2023-05-13 12:05:00,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:05:00,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 14: [2023-05-13 12:05:00,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 26: [2023-05-13 12:05:00,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:05:00,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 29: [2023-05-13 12:05:00,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 7: [2023-05-13 12:05:00,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,282] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,283] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,283] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:05:00,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:05:00,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:05:00,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 2: [2023-05-13 12:05:00,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 17: [2023-05-13 12:05:00,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 8: [2023-05-13 12:05:00,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 17: [2023-05-13 12:05:00,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 13: [2023-05-13 12:05:00,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt... 11: [2023-05-13 12:05:00,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 15: [2023-05-13 12:05:00,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:05:00,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 20: [2023-05-13 12:05:00,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 2: [2023-05-13 12:05:00,319] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,319] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 27: [2023-05-13 12:05:00,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,323] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,326] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 8: [2023-05-13 12:05:00,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 6: [2023-05-13 12:05:00,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 15: [2023-05-13 12:05:00,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 15: [2023-05-13 12:05:00,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 0: [2023-05-13 12:05:00,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 17: [2023-05-13 12:05:00,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 1: [2023-05-13 12:05:00,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 17: [2023-05-13 12:05:00,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_09-model_00-model_states.pt. 14: [2023-05-13 12:05:00,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 20: [2023-05-13 12:05:00,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 20: [2023-05-13 12:05:00,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 24: [2023-05-13 12:05:00,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 24: [2023-05-13 12:05:00,453] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,460] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,460] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,460] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,460] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,460] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,468] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,468] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 4: [2023-05-13 12:05:00,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 10: [2023-05-13 12:05:00,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,473] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,473] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,473] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:05:00,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 23: [2023-05-13 12:05:00,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 16: [2023-05-13 12:05:00,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,487] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 25: [2023-05-13 12:05:00,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 18: [2023-05-13 12:05:00,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 25: [2023-05-13 12:05:00,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 9: [2023-05-13 12:05:00,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 9: [2023-05-13 12:05:00,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,525] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,525] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,533] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 5: [2023-05-13 12:05:00,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,543] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:00,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:00,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,561] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:00,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 4: [2023-05-13 12:05:00,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 23: [2023-05-13 12:05:00,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 10: [2023-05-13 12:05:00,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:00,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 16: [2023-05-13 12:05:00,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,584] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:00,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 12: [2023-05-13 12:05:00,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:00,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 29: [2023-05-13 12:05:00,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 18: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 3: [2023-05-13 12:05:00,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:00,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 22: [2023-05-13 12:05:00,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 28: [2023-05-13 12:05:00,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 3: [2023-05-13 12:05:00,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 7: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 15: [2023-05-13 12:05:00,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:00,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:00,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 15: [2023-05-13 12:05:00,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 5: [2023-05-13 12:05:00,610] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 5: [2023-05-13 12:05:00,610] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:00,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:00,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 21: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:00,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 14: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:00,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 2: [2023-05-13 12:05:00,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 27: [2023-05-13 12:05:00,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 14: [2023-05-13 12:05:00,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 19: [2023-05-13 12:05:00,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 13: [2023-05-13 12:05:00,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 21: [2023-05-13 12:05:00,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:00,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:00,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 31: [2023-05-13 12:05:00,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:00,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,632] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:00,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 1: [2023-05-13 12:05:00,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 11: [2023-05-13 12:05:00,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:00,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 26: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:05:00,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:00,642] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 5: [2023-05-13 12:05:00,644] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:00,644] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:00,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:00,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 31: [2023-05-13 12:05:00,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:00,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 2: [2023-05-13 12:05:00,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:00,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:00,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:00,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:00,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:00,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:00,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,658] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,658] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,658] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:00,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 30: [2023-05-13 12:05:00,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 31: [2023-05-13 12:05:00,662] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:00,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:00,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:00,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 31: [2023-05-13 12:05:00,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:00,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 15: [2023-05-13 12:05:00,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:00,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 8: [2023-05-13 12:05:00,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:00,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 0: [2023-05-13 12:05:00,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 17: [2023-05-13 12:05:00,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt... 6: [2023-05-13 12:05:00,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 31: [2023-05-13 12:05:00,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 31: [2023-05-13 12:05:00,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:00,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 15: [2023-05-13 12:05:00,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:00,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 29: [2023-05-13 12:05:00,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 7: [2023-05-13 12:05:00,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:00,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 30: [2023-05-13 12:05:00,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:00,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 28: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 15: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:00,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:00,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,692] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:00,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 15: [2023-05-13 12:05:00,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 22: [2023-05-13 12:05:00,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:00,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:00,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 19: [2023-05-13 12:05:00,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 11: [2023-05-13 12:05:00,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:00,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:00,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 8: [2023-05-13 12:05:00,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:00,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:00,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:00,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,714] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 27: [2023-05-13 12:05:00,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:00,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:00,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:00,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:00,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 26: [2023-05-13 12:05:00,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 13: [2023-05-13 12:05:00,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 15: [2023-05-13 12:05:00,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:00,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:00,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:00,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:00,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:00,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:00,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:00,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 14: [2023-05-13 12:05:00,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:00,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:00,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:00,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:00,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 7: [2023-05-13 12:05:00,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:00,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:00,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:00,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:00,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:00,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:00,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 14: [2023-05-13 12:05:00,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:00,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:00,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,744] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:00,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:00,748] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,748] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,748] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,748] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 0: [2023-05-13 12:05:00,748] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 6: [2023-05-13 12:05:00,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:00,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:00,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 17: [2023-05-13 12:05:00,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_10-model_00-model_states.pt. 1: [2023-05-13 12:05:00,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:00,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:00,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:00,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:00,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 17: [2023-05-13 12:05:00,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 17: [2023-05-13 12:05:00,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,753] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,753] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,753] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 24: [2023-05-13 12:05:00,753] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,747] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,747] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,747] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:00,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:00,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:00,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:00,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:00,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:00,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:00,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:00,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:00,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:00,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 24: [2023-05-13 12:05:00,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:00,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:00,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 17: [2023-05-13 12:05:00,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 17: [2023-05-13 12:05:00,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 17: [2023-05-13 12:05:00,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 17: [2023-05-13 12:05:00,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 17: [2023-05-13 12:05:00,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 20: [2023-05-13 12:05:00,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 20: [2023-05-13 12:05:00,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:00,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:00,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:00,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:00,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:00,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:00,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 25: [2023-05-13 12:05:00,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 9: [2023-05-13 12:05:00,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 18: [2023-05-13 12:05:00,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,827] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,827] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,827] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,850] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 23: [2023-05-13 12:05:00,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:00,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:00,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 9: [2023-05-13 12:05:00,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:00,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 23: [2023-05-13 12:05:00,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:00,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 4: [2023-05-13 12:05:00,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 4: [2023-05-13 12:05:00,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 25: [2023-05-13 12:05:00,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:00,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:00,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:00,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:00,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:00,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:00,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:00,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:00,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:00,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:00,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:00,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:00,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:00,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:00,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:00,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:00,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:00,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:00,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 18: [2023-05-13 12:05:00,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:00,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:00,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,910] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 10: [2023-05-13 12:05:00,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 27: [2023-05-13 12:05:00,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:00,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:00,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:00,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:00,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:00,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:00,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:00,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:00,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:00,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:00,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:00,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:00,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:00,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:00,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:00,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:00,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:00,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:00,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:00,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:00,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:00,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:00,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:00,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:00,949] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:00,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:00,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 3: [2023-05-13 12:05:00,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:00,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:00,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:00,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 3: [2023-05-13 12:05:00,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:00,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:00,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:00,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:00,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:00,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 10: [2023-05-13 12:05:00,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:00,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 30: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:00,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:00,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:00,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 19: [2023-05-13 12:05:00,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:00,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:00,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:00,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:00,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:00,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:00,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:00,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:00,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:00,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:00,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:00,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 30: [2023-05-13 12:05:00,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:00,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:00,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:00,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:00,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:00,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:00,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:00,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:00,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 2: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:00,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:00,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:00,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:00,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 8: [2023-05-13 12:05:00,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:00,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:00,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:00,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:00,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:00,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:00,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:00,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 11: [2023-05-13 12:05:00,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:00,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,996] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,996] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,996] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:00,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:00,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:00,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:00,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:00,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:01,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:01,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:01,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:01,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:01,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:01,005] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:01,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:01,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:01,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:01,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:01,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:01,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 29: [2023-05-13 12:05:01,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:01,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:01,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:00,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:00,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 27: [2023-05-13 12:05:01,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:01,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 16: [2023-05-13 12:05:01,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,021] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:01,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,023] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:01,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:01,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:01,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:01,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:01,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,028] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:01,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:01,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 12: [2023-05-13 12:05:01,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:01,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:01,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:01,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 29: [2023-05-13 12:05:01,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:01,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:01,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:01,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:01,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,040] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:01,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:01,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 2: [2023-05-13 12:05:01,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:01,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 16: [2023-05-13 12:05:01,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 19: [2023-05-13 12:05:01,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 8: [2023-05-13 12:05:01,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:01,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 28: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:01,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:01,047] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 12: [2023-05-13 12:05:01,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:01,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 13: [2023-05-13 12:05:01,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:01,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:01,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:01,051] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 26: [2023-05-13 12:05:01,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 11: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 21: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:01,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:01,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:01,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,056] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:01,056] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:01,056] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:01,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:01,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:01,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:01,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 22: [2023-05-13 12:05:01,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:01,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:01,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:01,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:01,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:01,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:01,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:01,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:01,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:01,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:01,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:01,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:01,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:01,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:01,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 6: [2023-05-13 12:05:01,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:01,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:01,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:01,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 28: [2023-05-13 12:05:01,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 21: [2023-05-13 12:05:01,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:01,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 21: [2023-05-13 12:05:01,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:01,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 7: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 7: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 7: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 7: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,073] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:01,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:01,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:01,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:01,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 0: [2023-05-13 12:05:01,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 1: [2023-05-13 12:05:01,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 22: [2023-05-13 12:05:01,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:01,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 13: [2023-05-13 12:05:01,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:01,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:01,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:01,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:01,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt... 26: [2023-05-13 12:05:01,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:01,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:01,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:01,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:01,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:01,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:01,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:01,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:01,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:01,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 8: [2023-05-13 12:05:01,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 11: [2023-05-13 12:05:01,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,101] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:01,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,123] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:01,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:01,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 6: [2023-05-13 12:05:01,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 0: [2023-05-13 12:05:01,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,153] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,154] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,154] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,154] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 7: [2023-05-13 12:05:01,156] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 7: [2023-05-13 12:05:01,156] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 6: [2023-05-13 12:05:01,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,163] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 7: [2023-05-13 12:05:01,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 7: [2023-05-13 12:05:01,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 7: [2023-05-13 12:05:01,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 7: [2023-05-13 12:05:01,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,175] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:01,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 6: [2023-05-13 12:05:01,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:01,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:01,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:01,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:01,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 1: [2023-05-13 12:05:01,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_11-model_00-model_states.pt. 17: [2023-05-13 12:05:01,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 31: [2023-05-13 12:05:01,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 31: [2023-05-13 12:05:01,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 31: [2023-05-13 12:05:01,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 31: [2023-05-13 12:05:01,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 31: [2023-05-13 12:05:01,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 31: [2023-05-13 12:05:01,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 25: [2023-05-13 12:05:01,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 7: [2023-05-13 12:05:01,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 5: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:01,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:01,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 20: [2023-05-13 12:05:01,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:01,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:01,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,266] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,266] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,266] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 23: [2023-05-13 12:05:01,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,267] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,267] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 23: [2023-05-13 12:05:01,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:01,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:01,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:01,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:01,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:01,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:01,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 24: [2023-05-13 12:05:01,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 14: [2023-05-13 12:05:01,282] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,282] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:01,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 31: [2023-05-13 12:05:01,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:01,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:01,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 5: [2023-05-13 12:05:01,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:01,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 24: [2023-05-13 12:05:01,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,308] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,308] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,308] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 25: [2023-05-13 12:05:01,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 31: [2023-05-13 12:05:01,320] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,324] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,324] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,325] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:01,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:01,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:01,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:01,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:01,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:01,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:01,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:01,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 14: [2023-05-13 12:05:01,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:01,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:01,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 4: [2023-05-13 12:05:01,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:01,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:01,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 4: [2023-05-13 12:05:01,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:01,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:01,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 10: [2023-05-13 12:05:01,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 14: [2023-05-13 12:05:01,364] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,364] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,364] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 14: [2023-05-13 12:05:01,364] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 10: [2023-05-13 12:05:01,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 14: [2023-05-13 12:05:01,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 14: [2023-05-13 12:05:01,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,382] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,384] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 9: [2023-05-13 12:05:01,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:01,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:01,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 14: [2023-05-13 12:05:01,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 9: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 14: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:01,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 13: [2023-05-13 12:05:01,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 14: [2023-05-13 12:05:01,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 14: [2023-05-13 12:05:01,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 29: [2023-05-13 12:05:01,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:01,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:01,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:01,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:01,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:01,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:01,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 3: [2023-05-13 12:05:01,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 30: [2023-05-13 12:05:01,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:01,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:01,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:01,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:01,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 28: [2023-05-13 12:05:01,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 19: [2023-05-13 12:05:01,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:01,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 18: [2023-05-13 12:05:01,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:01,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:01,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:01,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:01,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 18: [2023-05-13 12:05:01,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 15: [2023-05-13 12:05:01,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 17: [2023-05-13 12:05:01,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:01,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 22: [2023-05-13 12:05:01,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 29: [2023-05-13 12:05:01,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,437] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,437] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 30: [2023-05-13 12:05:01,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,443] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:01,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 27: [2023-05-13 12:05:01,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 12: [2023-05-13 12:05:01,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,453] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:01,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 1: [2023-05-13 12:05:01,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 0: [2023-05-13 12:05:01,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 26: [2023-05-13 12:05:01,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:01,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 2: [2023-05-13 12:05:01,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt... 16: [2023-05-13 12:05:01,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,468] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 15: [2023-05-13 12:05:01,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:01,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:01,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 26: [2023-05-13 12:05:01,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 11: [2023-05-13 12:05:01,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:01,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:01,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:01,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,487] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,487] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 2: [2023-05-13 12:05:01,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:01,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 3: [2023-05-13 12:05:01,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:01,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:01,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:01,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 2: [2023-05-13 12:05:01,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:01,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 13: [2023-05-13 12:05:01,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 28: [2023-05-13 12:05:01,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 19: [2023-05-13 12:05:01,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:01,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,507] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,507] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,507] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,507] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 16: [2023-05-13 12:05:01,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,507] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 12: [2023-05-13 12:05:01,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 27: [2023-05-13 12:05:01,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:01,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 8: [2023-05-13 12:05:01,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 17: [2023-05-13 12:05:01,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 22: [2023-05-13 12:05:01,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 1: [2023-05-13 12:05:01,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 21: [2023-05-13 12:05:01,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 0: [2023-05-13 12:05:01,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_12-model_00-model_states.pt. 20: [2023-05-13 12:05:01,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:01,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:01,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:01,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,569] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,569] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:01,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:01,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:01,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:01,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 21: [2023-05-13 12:05:01,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 7: [2023-05-13 12:05:01,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,610] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 21: [2023-05-13 12:05:01,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 21: [2023-05-13 12:05:01,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 21: [2023-05-13 12:05:01,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 7: [2023-05-13 12:05:01,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:01,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:01,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 23: [2023-05-13 12:05:01,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,664] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:01,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:01,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 24: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:01,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 23: [2023-05-13 12:05:01,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:01,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:01,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:01,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:01,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:01,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:01,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,692] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,692] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,714] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,720] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,720] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,720] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,720] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,720] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,720] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,720] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 25: [2023-05-13 12:05:01,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:01,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:01,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,728] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 25: [2023-05-13 12:05:01,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:01,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:01,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:01,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:01,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:01,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:01,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:01,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:01,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:01,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:01,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 6: [2023-05-13 12:05:01,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 6: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:01,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:01,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:01,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:01,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:01,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:01,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 31: [2023-05-13 12:05:01,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:01,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:01,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:01,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 31: [2023-05-13 12:05:01,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:01,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 5: [2023-05-13 12:05:01,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:01,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:01,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:01,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:01,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:01,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:01,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:01,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:01,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:01,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 5: [2023-05-13 12:05:01,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:01,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:01,802] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 10: [2023-05-13 12:05:01,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:01,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:01,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:01,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 10: [2023-05-13 12:05:01,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:01,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:01,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:01,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:01,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:01,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:01,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:01,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,833] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,842] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,842] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,844] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:01,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:01,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:01,846] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,846] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:01,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 29: [2023-05-13 12:05:01,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 9: [2023-05-13 12:05:01,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:01,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 15: [2023-05-13 12:05:01,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:01,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 15: [2023-05-13 12:05:01,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:01,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 9: [2023-05-13 12:05:01,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:01,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 3: [2023-05-13 12:05:01,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 4: [2023-05-13 12:05:01,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:01,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 30: [2023-05-13 12:05:01,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 18: [2023-05-13 12:05:01,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 30: [2023-05-13 12:05:01,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:01,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:01,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:01,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 18: [2023-05-13 12:05:01,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:01,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:01,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:01,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:01,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:01,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:01,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,876] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,876] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:01,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:01,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:01,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 4: [2023-05-13 12:05:01,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 8: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:01,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:01,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:01,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:01,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:01,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 0: [2023-05-13 12:05:01,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 11: [2023-05-13 12:05:01,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:01,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 13: [2023-05-13 12:05:01,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:01,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 22: [2023-05-13 12:05:01,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:01,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:01,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 13: [2023-05-13 12:05:01,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:01,908] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,909] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,909] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 26: [2023-05-13 12:05:01,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 27: [2023-05-13 12:05:01,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 14: [2023-05-13 12:05:01,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 11: [2023-05-13 12:05:01,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:01,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:01,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:01,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:01,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:01,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:01,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 22: [2023-05-13 12:05:01,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:01,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:01,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 20: [2023-05-13 12:05:01,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:01,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:01,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:01,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:01,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:01,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:01,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:01,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 12: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:01,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:01,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:01,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:01,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 21: [2023-05-13 12:05:01,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 19: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 16: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 21: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:01,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:01,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:01,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 21: [2023-05-13 12:05:01,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 21: [2023-05-13 12:05:01,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 21: [2023-05-13 12:05:01,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:01,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:01,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 12: [2023-05-13 12:05:01,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:01,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 11: [2023-05-13 12:05:01,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:01,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:01,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:01,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:01,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:01,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:01,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:01,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:01,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:01,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:01,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:01,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:01,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:01,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:01,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 29: [2023-05-13 12:05:01,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:01,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:01,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 8: [2023-05-13 12:05:01,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:01,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 3: [2023-05-13 12:05:01,949] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:01,949] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:01,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:01,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:01,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:01,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:01,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 17: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:01,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:01,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:01,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:01,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:01,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:01,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 12: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 19: [2023-05-13 12:05:01,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:01,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 12: [2023-05-13 12:05:01,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:01,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:01,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:01,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:01,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:01,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:01,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:01,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:01,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:01,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:01,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 21: [2023-05-13 12:05:01,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:01,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 27: [2023-05-13 12:05:01,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 28: [2023-05-13 12:05:01,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:01,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 0: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:01,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:01,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 16: [2023-05-13 12:05:01,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:01,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 28: [2023-05-13 12:05:01,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:01,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:01,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:01,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:01,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 1: [2023-05-13 12:05:01,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt... 17: [2023-05-13 12:05:01,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 8: [2023-05-13 12:05:01,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:01,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:01,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:01,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:01,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:01,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:01,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:01,986] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:01,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 21: [2023-05-13 12:05:01,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:01,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:01,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:01,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:01,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:01,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:01,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 14: [2023-05-13 12:05:01,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 26: [2023-05-13 12:05:01,996] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:01,996] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:01,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:01,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:01,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:01,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:01,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:01,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:01,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:01,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:01,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:01,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:02,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:02,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:02,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:02,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:02,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 20: [2023-05-13 12:05:02,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:02,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:02,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:02,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:02,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:02,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:02,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:02,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:02,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:02,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:02,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 7: [2023-05-13 12:05:02,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:02,013] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:02,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:02,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:02,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:02,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:02,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:02,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:02,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 21: [2023-05-13 12:05:02,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:02,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:02,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:02,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:02,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:02,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 2: [2023-05-13 12:05:02,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 20: [2023-05-13 12:05:02,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:02,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 2: [2023-05-13 12:05:02,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:02,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:02,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:02,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:02,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 23: [2023-05-13 12:05:02,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:02,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:02,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:02,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 1: [2023-05-13 12:05:02,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_13-model_00-model_states.pt. 24: [2023-05-13 12:05:02,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,080] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,080] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,080] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,080] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,080] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,080] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,080] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:02,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:02,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:02,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:02,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:02,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:02,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:02,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:02,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,112] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 6: [2023-05-13 12:05:02,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,134] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:02,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:02,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 5: [2023-05-13 12:05:02,136] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,136] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:02,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:02,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:02,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,154] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,162] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,162] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,162] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,168] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,170] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,175] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 6: [2023-05-13 12:05:02,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 30: [2023-05-13 12:05:02,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,198] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 5: [2023-05-13 12:05:02,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:02,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:02,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:02,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:02,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:02,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:02,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:02,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 30: [2023-05-13 12:05:02,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:02,207] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:02,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:02,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:02,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 15: [2023-05-13 12:05:02,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 31: [2023-05-13 12:05:02,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:02,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:02,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:02,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:02,217] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:02,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:02,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:02,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:02,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:02,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,250] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,250] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,250] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 9: [2023-05-13 12:05:02,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,267] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 18: [2023-05-13 12:05:02,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 30: [2023-05-13 12:05:02,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 30: [2023-05-13 12:05:02,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 30: [2023-05-13 12:05:02,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 30: [2023-05-13 12:05:02,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:02,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 9: [2023-05-13 12:05:02,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 10: [2023-05-13 12:05:02,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 4: [2023-05-13 12:05:02,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,308] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:02,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 22: [2023-05-13 12:05:02,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:02,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:02,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 10: [2023-05-13 12:05:02,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 19: [2023-05-13 12:05:02,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 22: [2023-05-13 12:05:02,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 18: [2023-05-13 12:05:02,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 4: [2023-05-13 12:05:02,320] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,325] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 15: [2023-05-13 12:05:02,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:02,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:02,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:02,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:02,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:02,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 25: [2023-05-13 12:05:02,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 25: [2023-05-13 12:05:02,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,362] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 13: [2023-05-13 12:05:02,362] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,367] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,367] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,367] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,367] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,367] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,367] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,367] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,368] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:02,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:02,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:02,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 28: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 26: [2023-05-13 12:05:02,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 0: [2023-05-13 12:05:02,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,374] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 29: [2023-05-13 12:05:02,382] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 29: [2023-05-13 12:05:02,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 28: [2023-05-13 12:05:02,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 19: [2023-05-13 12:05:02,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,398] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:02,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:02,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:02,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:02,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:02,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 27: [2023-05-13 12:05:02,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 3: [2023-05-13 12:05:02,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 3: [2023-05-13 12:05:02,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 20: [2023-05-13 12:05:02,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,418] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 11: [2023-05-13 12:05:02,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,426] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,426] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,426] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 23: [2023-05-13 12:05:02,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 2: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,430] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:02,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,434] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,434] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 8: [2023-05-13 12:05:02,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,443] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,443] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 17: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 7: [2023-05-13 12:05:02,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 14: [2023-05-13 12:05:02,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,453] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,453] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 0: [2023-05-13 12:05:02,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 24: [2023-05-13 12:05:02,455] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 14: [2023-05-13 12:05:02,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 16: [2023-05-13 12:05:02,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 24: [2023-05-13 12:05:02,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 16: [2023-05-13 12:05:02,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 1: [2023-05-13 12:05:02,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt... 17: [2023-05-13 12:05:02,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 26: [2023-05-13 12:05:02,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 27: [2023-05-13 12:05:02,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 16: [2023-05-13 12:05:02,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 11: [2023-05-13 12:05:02,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:02,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,487] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 23: [2023-05-13 12:05:02,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,492] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,492] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,492] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:02,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,494] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,494] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 2: [2023-05-13 12:05:02,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 15: [2023-05-13 12:05:02,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:02,507] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 12: [2023-05-13 12:05:02,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 15: [2023-05-13 12:05:02,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,479] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,479] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:02,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 6: [2023-05-13 12:05:02,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 15: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 30: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 30: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 30: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 11: [2023-05-13 12:05:02,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:02,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:02,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:02,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 31: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 23: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 10: [2023-05-13 12:05:02,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 12: [2023-05-13 12:05:02,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 7: [2023-05-13 12:05:02,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 20: [2023-05-13 12:05:02,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:02,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:02,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 24: [2023-05-13 12:05:02,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 2: [2023-05-13 12:05:02,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:02,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 8: [2023-05-13 12:05:02,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 21: [2023-05-13 12:05:02,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:02,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 21: [2023-05-13 12:05:02,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:02,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 13: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 8: [2023-05-13 12:05:02,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:02,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:02,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:02,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 7: [2023-05-13 12:05:02,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 6: [2023-05-13 12:05:02,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 25: [2023-05-13 12:05:02,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 20: [2023-05-13 12:05:02,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 6: [2023-05-13 12:05:02,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:02,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 6: [2023-05-13 12:05:02,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:02,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 24: [2023-05-13 12:05:02,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,555] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 1: [2023-05-13 12:05:02,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_14-model_00-model_states.pt. 31: [2023-05-13 12:05:02,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:02,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 31: [2023-05-13 12:05:02,561] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 5: [2023-05-13 12:05:02,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,569] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,575] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,575] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 9: [2023-05-13 12:05:02,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 15: [2023-05-13 12:05:02,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 9: [2023-05-13 12:05:02,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:02,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 30: [2023-05-13 12:05:02,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:02,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:02,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:02,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 4: [2023-05-13 12:05:02,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 5: [2023-05-13 12:05:02,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:02,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:02,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 30: [2023-05-13 12:05:02,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:02,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,612] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,612] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:02,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:02,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 18: [2023-05-13 12:05:02,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:02,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 10: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:02,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 22: [2023-05-13 12:05:02,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:02,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:02,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 15: [2023-05-13 12:05:02,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:02,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,632] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,632] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 25: [2023-05-13 12:05:02,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:02,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 15: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 15: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 22: [2023-05-13 12:05:02,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 15: [2023-05-13 12:05:02,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 4: [2023-05-13 12:05:02,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 26: [2023-05-13 12:05:02,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 13: [2023-05-13 12:05:02,644] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:02,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 18: [2023-05-13 12:05:02,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:02,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 13: [2023-05-13 12:05:02,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:02,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,661] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,662] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,662] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,663] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,663] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 19: [2023-05-13 12:05:02,664] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 29: [2023-05-13 12:05:02,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 3: [2023-05-13 12:05:02,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:02,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:02,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:02,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:02,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:02,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 29: [2023-05-13 12:05:02,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:02,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:02,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 27: [2023-05-13 12:05:02,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 28: [2023-05-13 12:05:02,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:02,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 3: [2023-05-13 12:05:02,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:02,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:02,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:02,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:02,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 26: [2023-05-13 12:05:02,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:02,728] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:02,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:02,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:02,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 17: [2023-05-13 12:05:02,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 28: [2023-05-13 12:05:02,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,744] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,744] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,744] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 0: [2023-05-13 12:05:02,744] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 14: [2023-05-13 12:05:02,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 19: [2023-05-13 12:05:02,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:02,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:02,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:02,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:02,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:02,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:02,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:02,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 17: [2023-05-13 12:05:02,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 0: [2023-05-13 12:05:02,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:02,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 1: [2023-05-13 12:05:02,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt... 14: [2023-05-13 12:05:02,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:02,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:02,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:02,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:02,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:02,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:02,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 27: [2023-05-13 12:05:02,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:02,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:02,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:02,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:02,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:02,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:02,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:02,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 1: [2023-05-13 12:05:02,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_15-model_00-model_states.pt. 16: [2023-05-13 12:05:02,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:02,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:02,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:02,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:02,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:02,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:02,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 16: [2023-05-13 12:05:02,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:02,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:02,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:02,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:02,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:02,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:02,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 16: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:02,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:02,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:02,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:02,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:02,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:02,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:02,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:02,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:02,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:02,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 6: [2023-05-13 12:05:02,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:02,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:02,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 6: [2023-05-13 12:05:02,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 23: [2023-05-13 12:05:02,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:02,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:02,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:02,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 6: [2023-05-13 12:05:02,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:02,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 10: [2023-05-13 12:05:02,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 6: [2023-05-13 12:05:02,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 9: [2023-05-13 12:05:02,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:02,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 6: [2023-05-13 12:05:02,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 6: [2023-05-13 12:05:02,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:02,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:02,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:02,989] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:02,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:02,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:02,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:02,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:02,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:02,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:02,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:02,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:02,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:02,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:02,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:02,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:03,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:03,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:03,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:03,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:03,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:03,005] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,005] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,005] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 11: [2023-05-13 12:05:03,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:03,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:03,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:03,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:03,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:03,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 31: [2023-05-13 12:05:03,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 2: [2023-05-13 12:05:03,008] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:03,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:03,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:03,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:03,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:03,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:03,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:03,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:03,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:03,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 11: [2023-05-13 12:05:03,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 11: [2023-05-13 12:05:03,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 11: [2023-05-13 12:05:03,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:03,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:03,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:03,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:03,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:03,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:03,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:03,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:03,023] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:03,023] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:03,029] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:03,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:03,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:03,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:03,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:03,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:03,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:03,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:03,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:03,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:03,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,038] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:03,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:03,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 2: [2023-05-13 12:05:03,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,040] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:03,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:03,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:03,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:03,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:03,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:03,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:03,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:03,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:03,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 9: [2023-05-13 12:05:03,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,051] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:03,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:03,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:03,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:03,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:03,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 6: [2023-05-13 12:05:03,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,058] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 31: [2023-05-13 12:05:03,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:03,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:03,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:03,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,061] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:03,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 23: [2023-05-13 12:05:03,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 7: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:03,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:03,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,073] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,073] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 12: [2023-05-13 12:05:03,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,073] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,073] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 10: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:03,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:03,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 5: [2023-05-13 12:05:03,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:03,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:03,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:03,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:03,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:03,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 21: [2023-05-13 12:05:03,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 12: [2023-05-13 12:05:03,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:03,080] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 21: [2023-05-13 12:05:03,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,087] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 8: [2023-05-13 12:05:03,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 24: [2023-05-13 12:05:03,101] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,101] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 24: [2023-05-13 12:05:03,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 7: [2023-05-13 12:05:03,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:03,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:03,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:03,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:03,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 5: [2023-05-13 12:05:03,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:03,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:03,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 8: [2023-05-13 12:05:03,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,134] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,142] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:03,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:03,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:03,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:03,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:03,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 0: [2023-05-13 12:05:03,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:03,152] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,153] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:03,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,154] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,154] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:03,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:03,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:03,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:03,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,162] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 29: [2023-05-13 12:05:03,163] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,163] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:03,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:03,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 4: [2023-05-13 12:05:03,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 29: [2023-05-13 12:05:03,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:03,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 15: [2023-05-13 12:05:03,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,184] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,184] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 15: [2023-05-13 12:05:03,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:03,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:03,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 15: [2023-05-13 12:05:03,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:03,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:03,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:03,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:03,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:03,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 22: [2023-05-13 12:05:03,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 26: [2023-05-13 12:05:03,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,198] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,199] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:03,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 3: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 3: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:03,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:03,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 17: [2023-05-13 12:05:03,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:03,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:03,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:03,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:03,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:03,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:03,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 19: [2023-05-13 12:05:03,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:03,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 13: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 18: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:03,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:03,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 28: [2023-05-13 12:05:03,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 25: [2023-05-13 12:05:03,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 30: [2023-05-13 12:05:03,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 27: [2023-05-13 12:05:03,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 1: [2023-05-13 12:05:03,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 1: [2023-05-13 12:05:03,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 1: [2023-05-13 12:05:03,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 1: [2023-05-13 12:05:03,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 1: [2023-05-13 12:05:03,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 1: [2023-05-13 12:05:03,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 1: [2023-05-13 12:05:03,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,225] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,225] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 20: [2023-05-13 12:05:03,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 27: [2023-05-13 12:05:03,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 20: [2023-05-13 12:05:03,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:03,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:03,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:03,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:03,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:03,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:03,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:03,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:03,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:03,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 1: [2023-05-13 12:05:03,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:03,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:03,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 14: [2023-05-13 12:05:03,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt... 30: [2023-05-13 12:05:03,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 25: [2023-05-13 12:05:03,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,256] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,256] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 4: [2023-05-13 12:05:03,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 0: [2023-05-13 12:05:03,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,269] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 26: [2023-05-13 12:05:03,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 22: [2023-05-13 12:05:03,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 15: [2023-05-13 12:05:03,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 19: [2023-05-13 12:05:03,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 17: [2023-05-13 12:05:03,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 18: [2023-05-13 12:05:03,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 1: [2023-05-13 12:05:03,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,319] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 13: [2023-05-13 12:05:03,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,335] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 1: [2023-05-13 12:05:03,335] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 28: [2023-05-13 12:05:03,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_16-model_00-model_states.pt. 14: [2023-05-13 12:05:03,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,360] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,393] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,393] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 10: [2023-05-13 12:05:03,398] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,453] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,468] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,487] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 23: [2023-05-13 12:05:03,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 5: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 11: [2023-05-13 12:05:03,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 5: [2023-05-13 12:05:03,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 6: [2023-05-13 12:05:03,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 11: [2023-05-13 12:05:03,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 11: [2023-05-13 12:05:03,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 11: [2023-05-13 12:05:03,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:03,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:03,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:03,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:03,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:03,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:03,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:03,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 23: [2023-05-13 12:05:03,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 9: [2023-05-13 12:05:03,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 7: [2023-05-13 12:05:03,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,569] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,569] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 16: [2023-05-13 12:05:03,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 11: [2023-05-13 12:05:03,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 11: [2023-05-13 12:05:03,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:03,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 16: [2023-05-13 12:05:03,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:03,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:03,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:03,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 8: [2023-05-13 12:05:03,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:03,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 8: [2023-05-13 12:05:03,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:03,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,642] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,643] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,644] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,648] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,648] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:03,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:03,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 20: [2023-05-13 12:05:03,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 20: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:03,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 3: [2023-05-13 12:05:03,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:03,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:03,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:03,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 3: [2023-05-13 12:05:03,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:03,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,687] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,687] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,687] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,687] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 17: [2023-05-13 12:05:03,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:03,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 27: [2023-05-13 12:05:03,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 21: [2023-05-13 12:05:03,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:03,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:03,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 4: [2023-05-13 12:05:03,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:03,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:03,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 21: [2023-05-13 12:05:03,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:03,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:03,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:03,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 22: [2023-05-13 12:05:03,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 14: [2023-05-13 12:05:03,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 27: [2023-05-13 12:05:03,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:03,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:03,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:03,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:03,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:03,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:03,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:03,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 20: [2023-05-13 12:05:03,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:03,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 0: [2023-05-13 12:05:03,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 12: [2023-05-13 12:05:03,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:03,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 24: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 18: [2023-05-13 12:05:03,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:03,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:03,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 1: [2023-05-13 12:05:03,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 4: [2023-05-13 12:05:03,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 20: [2023-05-13 12:05:03,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,747] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,747] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 19: [2023-05-13 12:05:03,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 15: [2023-05-13 12:05:03,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:03,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 24: [2023-05-13 12:05:03,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:03,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:03,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 12: [2023-05-13 12:05:03,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:03,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 28: [2023-05-13 12:05:03,757] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:03,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:03,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:03,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:03,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:03,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:03,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:03,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:03,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:03,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:03,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:03,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:03,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:03,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:03,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 0: [2023-05-13 12:05:03,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:03,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:03,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 20: [2023-05-13 12:05:03,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 20: [2023-05-13 12:05:03,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 20: [2023-05-13 12:05:03,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:03,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:03,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:03,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:03,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:03,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 25: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:03,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 22: [2023-05-13 12:05:03,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 26: [2023-05-13 12:05:03,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 26: [2023-05-13 12:05:03,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 30: [2023-05-13 12:05:03,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:03,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 29: [2023-05-13 12:05:03,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 31: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 18: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 25: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 13: [2023-05-13 12:05:03,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt... 2: [2023-05-13 12:05:03,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:03,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:03,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:03,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:03,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:03,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:03,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:03,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:03,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 17: [2023-05-13 12:05:03,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:03,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:03,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:03,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:03,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:03,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:03,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:03,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 2: [2023-05-13 12:05:03,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:03,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 14: [2023-05-13 12:05:03,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:03,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:03,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:03,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:03,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:03,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:03,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:03,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:03,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 29: [2023-05-13 12:05:03,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:03,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:03,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:03,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:03,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:03,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:03,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 28: [2023-05-13 12:05:03,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:03,835] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 15: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 10: [2023-05-13 12:05:03,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:03,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:03,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 7: [2023-05-13 12:05:03,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:03,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:03,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:03,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:03,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:03,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:03,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:03,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,845] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,845] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:03,845] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:03,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:03,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:03,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:03,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:03,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 31: [2023-05-13 12:05:03,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 19: [2023-05-13 12:05:03,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:03,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 9: [2023-05-13 12:05:03,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:03,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:03,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:03,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,872] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:03,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:03,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 1: [2023-05-13 12:05:03,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:03,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:03,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:03,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 13: [2023-05-13 12:05:03,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 6: [2023-05-13 12:05:03,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:03,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_17-model_00-model_states.pt. 30: [2023-05-13 12:05:03,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,888] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:03,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:03,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:03,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:03,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:03,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:03,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:03,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,908] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:03,910] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:03,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:03,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:03,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 2: [2023-05-13 12:05:03,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:03,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:03,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 2: [2023-05-13 12:05:03,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:03,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:03,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:03,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:03,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 6: [2023-05-13 12:05:03,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:03,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:03,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:03,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:03,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:03,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 5: [2023-05-13 12:05:03,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:03,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 10: [2023-05-13 12:05:03,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:03,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:03,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:03,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:03,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 31: [2023-05-13 12:05:03,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:03,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:03,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:03,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:03,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:03,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:03,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:03,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:03,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:03,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:03,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:03,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:03,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:03,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:03,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:03,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:03,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:03,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:03,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:03,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:03,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:03,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:03,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:03,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:03,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:03,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:03,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:03,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:03,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:03,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:03,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:03,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:03,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:03,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:03,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:03,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:03,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:03,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:03,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:03,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:03,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:03,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:03,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:03,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:03,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:03,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:03,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:03,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:03,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:03,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:03,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:03,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:03,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:03,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:03,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:03,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 6: [2023-05-13 12:05:03,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:03,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:03,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:03,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:04,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,013] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 11: [2023-05-13 12:05:04,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 11: [2023-05-13 12:05:04,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:04,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 3: [2023-05-13 12:05:04,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,066] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,075] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,075] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,075] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,075] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,075] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,075] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:04,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:04,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:04,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:04,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:04,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:04,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:04,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,089] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:04,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:04,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 12: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,102] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,102] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:04,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:04,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:04,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:04,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:04,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:04,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:04,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:04,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:04,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 26: [2023-05-13 12:05:04,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:04,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:04,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 8: [2023-05-13 12:05:04,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 21: [2023-05-13 12:05:04,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:04,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:04,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:04,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:04,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 21: [2023-05-13 12:05:04,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 21: [2023-05-13 12:05:04,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 21: [2023-05-13 12:05:04,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,145] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 21: [2023-05-13 12:05:04,153] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,158] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:04,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 26: [2023-05-13 12:05:04,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 12: [2023-05-13 12:05:04,168] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,168] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:04,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:04,170] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,170] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:04,170] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:04,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:04,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,173] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:04,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:04,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 27: [2023-05-13 12:05:04,175] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,175] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 22: [2023-05-13 12:05:04,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 22: [2023-05-13 12:05:04,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:04,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:04,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:04,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:04,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:04,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,225] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:04,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:04,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:04,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 14: [2023-05-13 12:05:04,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:04,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:04,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:04,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,238] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:04,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:04,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 29: [2023-05-13 12:05:04,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:04,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 16: [2023-05-13 12:05:04,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 24: [2023-05-13 12:05:04,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:04,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:04,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:04,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 0: [2023-05-13 12:05:04,250] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 4: [2023-05-13 12:05:04,250] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,250] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 20: [2023-05-13 12:05:04,250] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 20: [2023-05-13 12:05:04,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 20: [2023-05-13 12:05:04,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:04,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 20: [2023-05-13 12:05:04,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 0: [2023-05-13 12:05:04,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 24: [2023-05-13 12:05:04,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:04,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:04,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:04,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:04,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:04,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:04,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:04,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:04,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:04,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:04,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 1: [2023-05-13 12:05:04,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 16: [2023-05-13 12:05:04,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 28: [2023-05-13 12:05:04,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:04,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:04,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:04,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 17: [2023-05-13 12:05:04,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 4: [2023-05-13 12:05:04,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,267] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 15: [2023-05-13 12:05:04,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:04,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:04,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 15: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:04,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 25: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 18: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 23: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 7: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:04,282] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 23: [2023-05-13 12:05:04,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:04,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:04,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:04,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:04,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:04,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:04,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 7: [2023-05-13 12:05:04,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,299] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,299] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,299] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,299] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:04,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 28: [2023-05-13 12:05:04,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 25: [2023-05-13 12:05:04,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,320] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:04,320] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:04,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:04,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:04,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:04,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:04,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:04,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 13: [2023-05-13 12:05:04,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,325] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,326] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 29: [2023-05-13 12:05:04,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 5: [2023-05-13 12:05:04,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,335] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 20: [2023-05-13 12:05:04,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 13: [2023-05-13 12:05:04,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 10: [2023-05-13 12:05:04,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 18: [2023-05-13 12:05:04,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 9: [2023-05-13 12:05:04,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:04,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 14: [2023-05-13 12:05:04,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:04,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:04,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 17: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:04,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:04,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:04,360] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,361] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,364] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:04,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:04,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:04,374] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,374] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,374] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 9: [2023-05-13 12:05:04,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 10: [2023-05-13 12:05:04,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 2: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 23: [2023-05-13 12:05:04,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:04,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 23: [2023-05-13 12:05:04,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 30: [2023-05-13 12:05:04,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 1: [2023-05-13 12:05:04,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 10: [2023-05-13 12:05:04,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 5: [2023-05-13 12:05:04,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,398] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,398] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:04,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:04,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 7: [2023-05-13 12:05:04,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 2: [2023-05-13 12:05:04,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 30: [2023-05-13 12:05:04,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:04,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:04,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 19: [2023-05-13 12:05:04,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:04,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt... 9: [2023-05-13 12:05:04,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:04,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:04,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:04,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 31: [2023-05-13 12:05:04,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:04,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:04,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 27: [2023-05-13 12:05:04,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 27: [2023-05-13 12:05:04,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 31: [2023-05-13 12:05:04,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 31: [2023-05-13 12:05:04,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,479] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 19: [2023-05-13 12:05:04,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 8: [2023-05-13 12:05:04,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 21: [2023-05-13 12:05:04,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 21: [2023-05-13 12:05:04,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,492] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_18-model_00-model_states.pt. 3: [2023-05-13 12:05:04,492] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 21: [2023-05-13 12:05:04,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 21: [2023-05-13 12:05:04,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 21: [2023-05-13 12:05:04,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 21: [2023-05-13 12:05:04,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 8: [2023-05-13 12:05:04,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:04,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 26: [2023-05-13 12:05:04,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 3: [2023-05-13 12:05:04,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:04,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 3: [2023-05-13 12:05:04,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 11: [2023-05-13 12:05:04,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 11: [2023-05-13 12:05:04,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:04,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:04,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:04,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:04,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:04,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:04,533] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,533] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,533] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 6: [2023-05-13 12:05:04,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:04,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 6: [2023-05-13 12:05:04,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:04,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 27: [2023-05-13 12:05:04,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:04,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 21: [2023-05-13 12:05:04,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,576] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,576] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,584] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:04,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 8: [2023-05-13 12:05:04,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:04,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 26: [2023-05-13 12:05:04,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:04,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:04,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:04,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:04,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:04,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:04,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:04,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:04,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 16: [2023-05-13 12:05:04,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:04,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:04,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:04,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:04,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 16: [2023-05-13 12:05:04,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:04,642] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:04,642] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,643] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,643] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 29: [2023-05-13 12:05:04,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,663] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,664] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:04,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:04,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:04,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:04,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 12: [2023-05-13 12:05:04,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:04,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:04,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:04,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 4: [2023-05-13 12:05:04,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:04,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:04,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:04,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 22: [2023-05-13 12:05:04,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:04,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:04,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 4: [2023-05-13 12:05:04,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:04,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:04,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:04,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 12: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 28: [2023-05-13 12:05:04,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:04,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 22: [2023-05-13 12:05:04,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:04,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 20: [2023-05-13 12:05:04,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:04,723] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,723] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 13: [2023-05-13 12:05:04,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 13: [2023-05-13 12:05:04,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 1: [2023-05-13 12:05:04,728] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:04,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:04,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:04,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:04,731] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,731] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,731] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:04,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 20: [2023-05-13 12:05:04,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:04,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:04,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:04,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:04,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:04,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 14: [2023-05-13 12:05:04,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:04,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:04,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 28: [2023-05-13 12:05:04,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:04,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 29: [2023-05-13 12:05:04,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:04,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:04,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:04,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:04,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:04,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:04,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:04,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:04,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 24: [2023-05-13 12:05:04,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 24: [2023-05-13 12:05:04,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:04,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 25: [2023-05-13 12:05:04,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 18: [2023-05-13 12:05:04,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 30: [2023-05-13 12:05:04,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 19: [2023-05-13 12:05:04,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 15: [2023-05-13 12:05:04,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 0: [2023-05-13 12:05:04,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt... 17: [2023-05-13 12:05:04,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:04,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:04,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:04,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,803] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 7: [2023-05-13 12:05:04,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 9: [2023-05-13 12:05:04,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 15: [2023-05-13 12:05:04,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:04,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,820] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,827] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,833] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 25: [2023-05-13 12:05:04,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 17: [2023-05-13 12:05:04,833] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,837] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 18: [2023-05-13 12:05:04,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 1: [2023-05-13 12:05:04,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 19: [2023-05-13 12:05:04,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:04,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:04,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 5: [2023-05-13 12:05:04,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:04,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:04,850] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:04,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 14: [2023-05-13 12:05:04,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:04,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:04,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:04,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:04,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 0: [2023-05-13 12:05:04,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_19-model_00-model_states.pt. 30: [2023-05-13 12:05:04,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:04,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:04,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:04,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:04,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:04,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:04,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:04,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:04,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:04,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:04,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:04,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:04,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:04,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:04,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:04,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:04,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:04,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:04,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:04,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:04,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:04,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:04,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 5: [2023-05-13 12:05:04,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:04,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:04,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:04,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:04,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:04,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:04,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:04,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:04,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:04,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:04,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:04,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:04,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:04,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:04,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:04,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 31: [2023-05-13 12:05:04,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 5: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:04,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:04,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:04,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 8: [2023-05-13 12:05:04,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:04,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 11: [2023-05-13 12:05:04,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:04,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:04,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:04,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:04,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:04,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:04,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:04,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:04,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:04,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:04,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:04,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:04,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:04,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:04,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:04,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:04,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:04,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:04,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:04,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:04,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:04,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:04,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:04,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,989] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,989] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,989] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,989] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,989] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:04,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 10: [2023-05-13 12:05:04,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:04,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:04,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:04,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:04,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:04,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:04,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:04,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:04,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:05,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:05,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:05,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 27: [2023-05-13 12:05:05,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:05,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:05,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:05,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:05,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:05,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:05,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:05,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:05,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:05,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:05,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:05,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:05,008] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:05,008] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:05,008] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:05,008] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 10: [2023-05-13 12:05:05,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:05,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:05,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,028] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,028] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:05,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:05,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:05,033] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,033] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,033] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:05,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 11: [2023-05-13 12:05:05,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,044] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:05,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:05,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 8: [2023-05-13 12:05:05,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,047] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 31: [2023-05-13 12:05:05,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:05,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 21: [2023-05-13 12:05:05,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:05,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:05,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:05,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:05,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:05,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:05,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:05,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:05,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:05,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 3: [2023-05-13 12:05:05,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:05,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:05,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 6: [2023-05-13 12:05:05,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 21: [2023-05-13 12:05:05,078] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:05,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,088] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:05,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:05,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:05,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:05,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:05,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:05,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 26: [2023-05-13 12:05:05,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:05,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:05,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:05,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:05,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:05,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:05,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:05,096] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 16: [2023-05-13 12:05:05,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 6: [2023-05-13 12:05:05,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,160] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,162] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 20: [2023-05-13 12:05:05,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 3: [2023-05-13 12:05:05,168] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,168] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,170] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,173] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,175] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 26: [2023-05-13 12:05:05,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,190] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,190] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,191] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,191] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,191] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,191] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,191] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 16: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:05,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,198] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:05,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:05,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:05,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:05,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:05,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:05,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:05,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:05,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:05,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:05,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:05,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:05,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 20: [2023-05-13 12:05:05,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:05,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:05,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 13: [2023-05-13 12:05:05,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:05,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 13: [2023-05-13 12:05:05,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:05,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 9: [2023-05-13 12:05:05,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:05,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 13: [2023-05-13 12:05:05,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:05,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:05,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:05,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:05,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:05,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:05,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:05,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:05,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:05,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 25: [2023-05-13 12:05:05,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 15: [2023-05-13 12:05:05,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:05,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:05,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,256] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:05,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,256] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:05,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 2: [2023-05-13 12:05:05,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 24: [2023-05-13 12:05:05,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:05,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:05,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:05,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:05,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 30: [2023-05-13 12:05:05,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:05,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:05,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 19: [2023-05-13 12:05:05,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:05,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:05,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:05,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:05,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 7: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 22: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:05,267] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 17: [2023-05-13 12:05:05,267] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 14: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:05,267] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 4: [2023-05-13 12:05:05,268] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:05,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:05,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 18: [2023-05-13 12:05:05,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:05,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 23: [2023-05-13 12:05:05,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 28: [2023-05-13 12:05:05,271] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:05,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:05,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:05,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 4: [2023-05-13 12:05:05,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,282] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 15: [2023-05-13 12:05:05,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 12: [2023-05-13 12:05:05,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:05,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:05,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 1: [2023-05-13 12:05:05,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 13: [2023-05-13 12:05:05,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 12: [2023-05-13 12:05:05,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 29: [2023-05-13 12:05:05,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt... 0: [2023-05-13 12:05:05,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 29: [2023-05-13 12:05:05,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 22: [2023-05-13 12:05:05,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:05,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 28: [2023-05-13 12:05:05,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,308] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 25: [2023-05-13 12:05:05,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:05,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,319] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,319] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,320] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,323] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 27: [2023-05-13 12:05:05,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,325] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 0: [2023-05-13 12:05:05,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 2: [2023-05-13 12:05:05,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 13: [2023-05-13 12:05:05,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 19: [2023-05-13 12:05:05,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 13: [2023-05-13 12:05:05,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:05,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 24: [2023-05-13 12:05:05,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 9: [2023-05-13 12:05:05,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 30: [2023-05-13 12:05:05,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:05,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 13: [2023-05-13 12:05:05,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 2: [2023-05-13 12:05:05,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 2: [2023-05-13 12:05:05,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 23: [2023-05-13 12:05:05,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:05,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 7: [2023-05-13 12:05:05,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,360] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,360] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,366] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 18: [2023-05-13 12:05:05,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,369] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,369] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:05,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:05,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,374] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 14: [2023-05-13 12:05:05,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 1: [2023-05-13 12:05:05,385] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_20-model_00-model_states.pt. 17: [2023-05-13 12:05:05,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,398] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,398] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:05,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 5: [2023-05-13 12:05:05,426] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,443] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,443] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,443] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 10: [2023-05-13 12:05:05,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,449] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 11: [2023-05-13 12:05:05,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 8: [2023-05-13 12:05:05,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,460] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,460] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,468] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 31: [2023-05-13 12:05:05,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 8: [2023-05-13 12:05:05,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 10: [2023-05-13 12:05:05,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 16: [2023-05-13 12:05:05,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 3: [2023-05-13 12:05:05,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 3: [2023-05-13 12:05:05,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,569] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,569] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,583] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,583] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 16: [2023-05-13 12:05:05,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,610] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 21: [2023-05-13 12:05:05,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,644] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,658] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,663] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 20: [2023-05-13 12:05:05,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 21: [2023-05-13 12:05:05,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:05,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:05,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,692] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 2: [2023-05-13 12:05:05,692] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 2: [2023-05-13 12:05:05,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 6: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 6: [2023-05-13 12:05:05,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:05,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:05,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:05,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 19: [2023-05-13 12:05:05,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:05,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:05,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 20: [2023-05-13 12:05:05,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:05,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 4: [2023-05-13 12:05:05,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:05,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:05,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,728] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:05,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,728] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:05,731] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,731] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,731] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,731] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 7: [2023-05-13 12:05:05,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:05,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:05,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:05,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 0: [2023-05-13 12:05:05,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 17: [2023-05-13 12:05:05,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 9: [2023-05-13 12:05:05,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:05,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 1: [2023-05-13 12:05:05,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,744] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,744] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:05,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:05,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:05,747] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:05,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,748] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:05,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 13: [2023-05-13 12:05:05,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 15: [2023-05-13 12:05:05,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 7: [2023-05-13 12:05:05,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,753] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 29: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 2: [2023-05-13 12:05:05,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:05,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 2: [2023-05-13 12:05:05,757] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:05,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 13: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 13: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 13: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 9: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 29: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 12: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 29: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:05,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 2: [2023-05-13 12:05:05,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:05,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:05,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 27: [2023-05-13 12:05:05,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 22: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 26: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 26: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:05,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:05,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 4: [2023-05-13 12:05:05,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 25: [2023-05-13 12:05:05,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 27: [2023-05-13 12:05:05,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:05,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:05,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:05,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 9: [2023-05-13 12:05:05,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 7: [2023-05-13 12:05:05,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:05,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 19: [2023-05-13 12:05:05,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 24: [2023-05-13 12:05:05,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 2: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 9: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 9: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 9: [2023-05-13 12:05:05,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:05,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:05,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 30: [2023-05-13 12:05:05,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 14: [2023-05-13 12:05:05,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:05,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:05,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 2: [2023-05-13 12:05:05,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:05,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 18: [2023-05-13 12:05:05,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 23: [2023-05-13 12:05:05,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:05,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 28: [2023-05-13 12:05:05,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt... 22: [2023-05-13 12:05:05,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:05,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 2: [2023-05-13 12:05:05,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:05,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 2: [2023-05-13 12:05:05,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:05,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:05,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:05,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:05,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 7: [2023-05-13 12:05:05,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:05,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 2: [2023-05-13 12:05:05,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:05,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 7: [2023-05-13 12:05:05,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 7: [2023-05-13 12:05:05,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 7: [2023-05-13 12:05:05,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 7: [2023-05-13 12:05:05,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:05,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:05,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:05,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 24: [2023-05-13 12:05:05,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:05,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 27: [2023-05-13 12:05:05,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:05,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:05,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 12: [2023-05-13 12:05:05,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:05,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:05,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:05,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:05,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:05,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:05,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:05,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:05,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:05,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:05,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:05,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 5: [2023-05-13 12:05:05,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:05,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:05,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:05,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:05,802] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:05,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 27: [2023-05-13 12:05:05,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:05,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 28: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:05,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:05,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:05,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:05,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:05,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:05,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:05,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:05,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 29: [2023-05-13 12:05:05,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 14: [2023-05-13 12:05:05,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 23: [2023-05-13 12:05:05,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:05,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:05,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:05,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:05,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:05,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:05,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:05,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 31: [2023-05-13 12:05:05,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:05,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:05,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 23: [2023-05-13 12:05:05,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:05,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:05,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 23: [2023-05-13 12:05:05,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:05,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:05,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 0: [2023-05-13 12:05:05,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 15: [2023-05-13 12:05:05,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:05,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 11: [2023-05-13 12:05:05,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 5: [2023-05-13 12:05:05,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:05,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 17: [2023-05-13 12:05:05,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 25: [2023-05-13 12:05:05,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,837] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 30: [2023-05-13 12:05:05,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 29: [2023-05-13 12:05:05,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,842] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,842] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 13: [2023-05-13 12:05:05,842] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:05,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 18: [2023-05-13 12:05:05,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:05,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:05,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:05,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:05,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 1: [2023-05-13 12:05:05,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_21-model_00-model_states.pt. 29: [2023-05-13 12:05:05,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 3: [2023-05-13 12:05:05,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:05,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:05,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:05,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:05,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:05,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:05,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 5: [2023-05-13 12:05:05,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:05,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:05,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:05,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:05,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:05,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:05,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 5: [2023-05-13 12:05:05,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:05,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:05,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:05,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:05,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:05,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:05,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 10: [2023-05-13 12:05:05,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:05,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,893] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,893] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:05,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 11: [2023-05-13 12:05:05,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:05,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 31: [2023-05-13 12:05:05,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,908] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,908] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,908] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,909] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,909] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:05,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 10: [2023-05-13 12:05:05,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:05,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:05,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:05,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:05,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:05,918] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:05,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:05,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:05,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:05,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:05,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 11: [2023-05-13 12:05:05,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:05,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:05,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:05,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:05,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:05,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:05,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:05,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:05,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:05,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:05,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:05,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 31: [2023-05-13 12:05:05,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:05,945] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:05,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 16: [2023-05-13 12:05:05,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:05,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:05,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 3: [2023-05-13 12:05:05,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:05,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 16: [2023-05-13 12:05:05,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:05,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:05,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:05,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:05,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:05,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,953] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:05,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:05,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 21: [2023-05-13 12:05:05,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:05,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:05,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 21: [2023-05-13 12:05:05,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:05,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:05,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:05,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:05,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:05,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:05,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:05,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:05,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:05,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:05,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:05,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:05,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:05,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:05,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 8: [2023-05-13 12:05:05,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:05,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:05,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:05,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:05,993] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:05,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:05,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:05,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:05,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:05,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:05,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 8: [2023-05-13 12:05:05,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:05,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:05,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:05,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:05,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:05,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:06,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:06,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:06,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:06,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:06,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:06,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:06,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:06,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,014] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:06,018] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:06,019] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:06,021] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:06,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:06,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:06,029] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:06,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:06,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:06,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:06,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:06,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:06,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:06,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:06,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:06,032] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:06,033] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 20: [2023-05-13 12:05:06,033] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,033] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:06,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:06,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:06,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:06,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:06,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:06,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 20: [2023-05-13 12:05:06,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 26: [2023-05-13 12:05:06,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:06,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:06,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:06,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 6: [2023-05-13 12:05:06,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:06,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:06,051] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:06,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:06,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:06,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:06,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:06,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 29: [2023-05-13 12:05:06,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,060] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:06,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 6: [2023-05-13 12:05:06,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 29: [2023-05-13 12:05:06,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 29: [2023-05-13 12:05:06,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 29: [2023-05-13 12:05:06,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 29: [2023-05-13 12:05:06,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,066] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,066] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,066] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,066] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,066] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:06,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:06,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 12: [2023-05-13 12:05:06,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:06,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 26: [2023-05-13 12:05:06,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,074] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:06,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,075] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,075] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:06,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 28: [2023-05-13 12:05:06,077] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,077] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,077] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,077] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:06,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:06,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:06,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:06,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:06,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 14: [2023-05-13 12:05:06,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:06,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:06,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:06,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,086] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,088] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:06,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 4: [2023-05-13 12:05:06,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:06,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:06,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:06,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:06,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:06,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:06,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:06,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 18: [2023-05-13 12:05:06,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:06,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,098] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:06,099] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,102] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 12: [2023-05-13 12:05:06,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:06,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:06,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 25: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 28: [2023-05-13 12:05:06,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,114] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:06,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,117] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:06,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 22: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,117] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:06,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:06,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 30: [2023-05-13 12:05:06,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,120] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,120] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 22: [2023-05-13 12:05:06,120] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,121] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:06,120] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:06,121] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:06,121] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 15: [2023-05-13 12:05:06,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:06,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:06,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:06,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:06,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:06,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:06,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:06,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 17: [2023-05-13 12:05:06,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:06,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:06,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 1: [2023-05-13 12:05:06,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 19: [2023-05-13 12:05:06,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 24: [2023-05-13 12:05:06,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 0: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:06,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 13: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt... 24: [2023-05-13 12:05:06,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,131] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,131] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,134] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,134] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,138] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 0: [2023-05-13 12:05:06,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,140] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,151] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,151] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 15: [2023-05-13 12:05:06,153] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,156] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,156] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,156] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,156] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 29: [2023-05-13 12:05:06,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,163] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,168] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,170] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,174] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,174] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 18: [2023-05-13 12:05:06,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,174] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,175] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 19: [2023-05-13 12:05:06,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 4: [2023-05-13 12:05:06,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:06,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,186] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 14: [2023-05-13 12:05:06,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 25: [2023-05-13 12:05:06,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 30: [2023-05-13 12:05:06,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:06,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 1: [2023-05-13 12:05:06,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 17: [2023-05-13 12:05:06,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_22-model_00-model_states.pt. 13: [2023-05-13 12:05:06,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:06,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,381] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,381] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,381] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 21: [2023-05-13 12:05:06,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:06,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,476] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,476] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,476] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 31: [2023-05-13 12:05:06,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 11: [2023-05-13 12:05:06,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:06,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:06,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:06,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:06,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:06,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:06,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:06,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:06,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:06,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:06,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:06,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:06,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:06,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:06,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:06,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:06,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 31: [2023-05-13 12:05:06,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:06,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:06,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:06,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:06,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:06,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:06,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:06,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:06,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 21: [2023-05-13 12:05:06,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,561] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:06,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:06,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 21: [2023-05-13 12:05:06,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 21: [2023-05-13 12:05:06,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:06,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:06,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:06,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 20: [2023-05-13 12:05:06,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 3: [2023-05-13 12:05:06,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:06,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:06,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:06,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:06,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:06,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:06,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:06,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:06,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:06,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:06,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:06,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:06,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:06,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:06,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:06,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 28: [2023-05-13 12:05:06,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:06,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:06,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:06,625] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:06,632] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:06,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 23: [2023-05-13 12:05:06,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:06,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:06,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:06,643] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:06,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 8: [2023-05-13 12:05:06,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:06,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:06,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:06,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:06,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:06,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:06,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:06,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 10: [2023-05-13 12:05:06,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:06,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:06,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:06,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:06,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,663] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,664] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:06,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:06,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:06,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:06,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:06,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 23: [2023-05-13 12:05:06,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:06,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:06,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 2: [2023-05-13 12:05:06,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 2: [2023-05-13 12:05:06,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 2: [2023-05-13 12:05:06,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 2: [2023-05-13 12:05:06,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 9: [2023-05-13 12:05:06,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 2: [2023-05-13 12:05:06,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:06,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 2: [2023-05-13 12:05:06,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 9: [2023-05-13 12:05:06,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 9: [2023-05-13 12:05:06,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 9: [2023-05-13 12:05:06,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 9: [2023-05-13 12:05:06,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 9: [2023-05-13 12:05:06,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 6: [2023-05-13 12:05:06,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 5: [2023-05-13 12:05:06,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 5: [2023-05-13 12:05:06,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,684] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 5: [2023-05-13 12:05:06,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 5: [2023-05-13 12:05:06,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 5: [2023-05-13 12:05:06,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 5: [2023-05-13 12:05:06,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 5: [2023-05-13 12:05:06,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:06,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:06,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:06,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:06,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 22: [2023-05-13 12:05:06,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:06,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:06,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:06,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:06,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 22: [2023-05-13 12:05:06,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:06,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 27: [2023-05-13 12:05:06,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:06,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:06,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:06,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 2: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 0: [2023-05-13 12:05:06,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 2: [2023-05-13 12:05:06,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:06,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 27: [2023-05-13 12:05:06,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:06,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:06,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:06,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:06,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:06,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:06,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:06,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:06,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:06,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:06,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:06,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:06,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 0: [2023-05-13 12:05:06,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:06,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:06,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:06,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:06,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:06,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:06,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:06,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:06,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:06,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:06,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 9: [2023-05-13 12:05:06,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:06,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:06,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:06,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:06,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:06,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 29: [2023-05-13 12:05:06,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:06,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:06,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,812] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:06,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:06,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:06,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:06,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 1: [2023-05-13 12:05:06,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:06,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:06,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:06,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 24: [2023-05-13 12:05:06,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 12: [2023-05-13 12:05:06,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:06,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:06,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:06,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 14: [2023-05-13 12:05:06,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:06,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:06,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 7: [2023-05-13 12:05:06,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 7: [2023-05-13 12:05:06,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 7: [2023-05-13 12:05:06,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 7: [2023-05-13 12:05:06,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 13: [2023-05-13 12:05:06,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 29: [2023-05-13 12:05:06,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:06,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:06,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:06,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:06,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 26: [2023-05-13 12:05:06,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 7: [2023-05-13 12:05:06,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:06,892] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:06,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:06,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:06,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:06,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:06,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:06,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:06,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:06,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:06,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:06,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 24: [2023-05-13 12:05:06,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:06,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:06,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:06,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 21: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:06,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 26: [2023-05-13 12:05:06,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:06,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:06,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:06,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:06,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:06,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:06,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:06,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:06,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:06,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 21: [2023-05-13 12:05:06,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 21: [2023-05-13 12:05:06,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:06,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:06,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:06,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:06,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:06,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 21: [2023-05-13 12:05:06,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 21: [2023-05-13 12:05:06,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 21: [2023-05-13 12:05:06,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:06,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 18: [2023-05-13 12:05:06,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:06,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:06,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:06,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 16: [2023-05-13 12:05:06,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:06,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:06,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:06,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:06,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:06,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:06,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,948] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,949] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:06,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:06,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:06,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:06,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:06,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:06,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:06,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:06,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:06,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:06,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:06,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 15: [2023-05-13 12:05:06,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:06,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:06,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:06,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:06,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:06,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:06,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:06,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:06,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:06,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:06,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:06,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:06,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:06,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 18: [2023-05-13 12:05:06,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:06,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:06,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:06,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:06,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:06,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 25: [2023-05-13 12:05:06,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 15: [2023-05-13 12:05:06,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:06,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:06,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:06,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,986] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,986] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:06,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:06,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:06,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:06,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:06,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:06,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:06,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:06,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:06,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:06,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:06,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:06,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:06,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:06,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:06,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:06,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:06,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:06,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:06,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:06,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:07,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:07,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,003] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:07,003] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:07,003] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:07,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:07,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:07,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:07,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:07,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,009] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:07,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:07,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 21: [2023-05-13 12:05:07,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:07,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:07,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:07,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:07,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:07,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:07,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:07,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:07,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:07,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:07,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:07,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:07,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:07,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:07,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:07,018] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:07,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:07,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 3: [2023-05-13 12:05:07,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 3: [2023-05-13 12:05:07,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:07,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:07,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:07,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:07,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 21: [2023-05-13 12:05:07,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:07,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:07,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,026] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:07,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:07,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,029] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 21: [2023-05-13 12:05:07,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:07,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:07,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:07,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 25: [2023-05-13 12:05:07,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 20: [2023-05-13 12:05:07,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,030] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:07,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:07,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:07,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:07,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,033] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 16: [2023-05-13 12:05:07,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:07,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:07,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:07,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:07,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:07,035] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 19: [2023-05-13 12:05:07,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:07,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:07,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 28: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:07,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:07,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 6: [2023-05-13 12:05:07,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,038] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:07,038] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:07,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 14: [2023-05-13 12:05:07,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:07,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:07,040] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 19: [2023-05-13 12:05:07,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:07,040] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:07,047] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:07,047] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:07,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:07,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:07,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:07,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 0: [2023-05-13 12:05:07,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:07,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:07,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:07,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:07,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:07,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 20: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:07,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:07,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:07,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:07,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:07,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:07,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,067] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,068] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:07,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:07,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:07,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,073] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,075] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:07,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,077] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:07,077] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:07,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:07,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:07,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:07,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:07,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:07,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,085] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 11: [2023-05-13 12:05:07,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:07,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,088] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,088] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,092] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,092] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 10: [2023-05-13 12:05:07,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 10: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,094] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,094] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,094] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,095] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,095] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,096] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,096] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:07,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:07,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,097] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 30: [2023-05-13 12:05:07,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:07,098] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:07,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:07,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:07,101] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:07,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 5: [2023-05-13 12:05:07,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:07,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:07,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:07,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:07,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:07,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 23: [2023-05-13 12:05:07,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 8: [2023-05-13 12:05:07,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 27: [2023-05-13 12:05:07,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:07,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 2: [2023-05-13 12:05:07,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:07,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 11: [2023-05-13 12:05:07,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,111] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,111] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 27: [2023-05-13 12:05:07,112] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 23: [2023-05-13 12:05:07,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 2: [2023-05-13 12:05:07,114] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 8: [2023-05-13 12:05:07,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 0: [2023-05-13 12:05:07,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 8: [2023-05-13 12:05:07,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 11: [2023-05-13 12:05:07,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,120] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 28: [2023-05-13 12:05:07,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:07,121] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,121] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,121] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,122] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 2: [2023-05-13 12:05:07,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:07,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:07,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 30: [2023-05-13 12:05:07,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:07,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:07,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:07,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:07,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:07,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 2: [2023-05-13 12:05:07,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,142] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,145] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,145] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 0: [2023-05-13 12:05:07,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:07,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:07,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:07,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 14: [2023-05-13 12:05:07,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:07,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,151] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,152] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,153] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,155] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:07,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,163] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,163] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:07,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 0: [2023-05-13 12:05:07,175] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:07,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:07,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:07,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 9: [2023-05-13 12:05:07,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:07,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:07,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:07,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:07,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 9: [2023-05-13 12:05:07,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,184] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,184] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 0: [2023-05-13 12:05:07,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,191] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 0: [2023-05-13 12:05:07,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 0: [2023-05-13 12:05:07,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 0: [2023-05-13 12:05:07,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:07,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 12: [2023-05-13 12:05:07,207] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,209] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,209] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,209] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,209] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,209] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 13: [2023-05-13 12:05:07,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 13: [2023-05-13 12:05:07,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:07,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 17: [2023-05-13 12:05:07,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 7: [2023-05-13 12:05:07,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,217] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 5: [2023-05-13 12:05:07,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:07,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:07,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 5: [2023-05-13 12:05:07,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:07,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:07,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:07,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:07,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 7: [2023-05-13 12:05:07,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:07,223] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:07,223] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:07,223] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:07,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 22: [2023-05-13 12:05:07,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:07,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:07,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:07,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:07,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:07,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 1: [2023-05-13 12:05:07,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:07,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 29: [2023-05-13 12:05:07,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 12: [2023-05-13 12:05:07,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 12: [2023-05-13 12:05:07,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:07,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:07,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:07,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:07,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:07,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:07,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:07,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 24: [2023-05-13 12:05:07,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:07,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:07,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:07,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:07,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:07,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:07,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 15: [2023-05-13 12:05:07,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 22: [2023-05-13 12:05:07,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 29: [2023-05-13 12:05:07,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,283] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,283] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,283] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 18: [2023-05-13 12:05:07,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,292] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,297] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,297] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,297] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,297] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,299] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 18: [2023-05-13 12:05:07,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 26: [2023-05-13 12:05:07,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 19: [2023-05-13 12:05:07,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 26: [2023-05-13 12:05:07,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,319] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 18: [2023-05-13 12:05:07,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 18: [2023-05-13 12:05:07,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 18: [2023-05-13 12:05:07,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 18: [2023-05-13 12:05:07,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:07,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 24: [2023-05-13 12:05:07,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 15: [2023-05-13 12:05:07,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 19: [2023-05-13 12:05:07,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 25: [2023-05-13 12:05:07,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,370] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 1: [2023-05-13 12:05:07,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 25: [2023-05-13 12:05:07,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 21: [2023-05-13 12:05:07,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:07,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 21: [2023-05-13 12:05:07,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 21: [2023-05-13 12:05:07,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 21: [2023-05-13 12:05:07,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 21: [2023-05-13 12:05:07,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 21: [2023-05-13 12:05:07,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 21: [2023-05-13 12:05:07,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,435] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:07,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 30: [2023-05-13 12:05:07,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 20: [2023-05-13 12:05:07,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,448] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,448] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,448] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,448] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,448] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:07,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 16: [2023-05-13 12:05:07,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 31: [2023-05-13 12:05:07,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,453] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,453] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,453] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,453] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,453] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 17: [2023-05-13 12:05:07,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 10: [2023-05-13 12:05:07,455] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,455] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 10: [2023-05-13 12:05:07,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,468] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 20: [2023-05-13 12:05:07,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:07,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:07,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:07,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:07,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:07,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,517] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,520] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,520] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 16: [2023-05-13 12:05:07,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:07,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:07,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:07,525] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:07,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:07,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 3: [2023-05-13 12:05:07,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 21: [2023-05-13 12:05:07,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:07,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:07,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:07,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:07,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 31: [2023-05-13 12:05:07,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 10: [2023-05-13 12:05:07,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:07,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:07,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:07,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:07,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:07,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:07,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 9: [2023-05-13 12:05:07,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 17: [2023-05-13 12:05:07,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:07,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:07,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:07,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 16: [2023-05-13 12:05:07,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 9: [2023-05-13 12:05:07,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:07,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:07,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:07,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:07,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:07,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:07,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:07,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:07,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:07,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,559] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,561] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,569] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,569] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:07,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:07,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:07,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,576] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 14: [2023-05-13 12:05:07,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,583] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:07,584] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 14: [2023-05-13 12:05:07,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:07,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 12: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 9: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:07,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:07,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 28: [2023-05-13 12:05:07,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 31: [2023-05-13 12:05:07,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:07,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 7: [2023-05-13 12:05:07,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:07,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:07,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 18: [2023-05-13 12:05:07,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 18: [2023-05-13 12:05:07,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:07,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:07,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 3: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 18: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 18: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:07,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:07,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 22: [2023-05-13 12:05:07,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:07,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:07,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,642] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,642] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 15: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,643] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 13: [2023-05-13 12:05:07,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,648] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,648] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,648] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,648] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:07,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:07,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,651] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:07,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 0: [2023-05-13 12:05:07,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 5: [2023-05-13 12:05:07,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 27: [2023-05-13 12:05:07,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 0: [2023-05-13 12:05:07,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,656] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:07,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:07,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 23: [2023-05-13 12:05:07,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:07,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,664] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:07,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 12: [2023-05-13 12:05:07,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:07,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:07,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:07,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:07,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:07,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:07,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:07,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 18: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:07,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:07,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:07,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:07,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 6: [2023-05-13 12:05:07,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:07,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:07,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:07,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:07,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 5: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 29: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 8: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:07,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 25: [2023-05-13 12:05:07,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:07,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:07,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 27: [2023-05-13 12:05:07,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 29: [2023-05-13 12:05:07,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:07,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 28: [2023-05-13 12:05:07,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 11: [2023-05-13 12:05:07,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:07,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:07,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 24: [2023-05-13 12:05:07,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:07,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:07,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:07,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 11: [2023-05-13 12:05:07,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:07,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:07,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:07,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:07,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:07,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:07,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 2: [2023-05-13 12:05:07,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:07,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:07,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 18: [2023-05-13 12:05:07,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:07,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:07,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:07,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:07,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:07,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 13: [2023-05-13 12:05:07,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:07,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:07,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:07,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:07,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:07,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:07,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:07,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:07,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:07,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:07,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:07,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:07,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:07,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 7: [2023-05-13 12:05:07,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:07,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:07,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:07,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:07,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:07,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:07,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:07,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,728] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:07,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:07,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 22: [2023-05-13 12:05:07,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:07,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 0: [2023-05-13 12:05:07,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 19: [2023-05-13 12:05:07,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 24: [2023-05-13 12:05:07,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 26: [2023-05-13 12:05:07,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:07,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:07,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:07,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:07,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 15: [2023-05-13 12:05:07,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 25: [2023-05-13 12:05:07,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:07,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:07,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:07,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:07,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 1: [2023-05-13 12:05:07,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:07,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:07,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 1: [2023-05-13 12:05:07,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:07,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:07,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:07,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:07,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:07,833] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:07,833] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:07,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:07,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 4: [2023-05-13 12:05:07,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt... 17: [2023-05-13 12:05:07,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,845] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,845] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 17: [2023-05-13 12:05:07,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 30: [2023-05-13 12:05:07,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 4: [2023-05-13 12:05:07,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_23-model_00-model_states.pt. 4: [2023-05-13 12:05:07,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 4: [2023-05-13 12:05:07,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 4: [2023-05-13 12:05:07,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:07,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:07,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 4: [2023-05-13 12:05:07,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 4: [2023-05-13 12:05:07,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:07,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 30: [2023-05-13 12:05:07,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:07,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 17: [2023-05-13 12:05:07,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:07,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:07,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:07,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:07,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:07,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:07,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:07,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:07,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:07,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:07,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:07,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:07,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:07,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:07,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:07,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:07,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:08,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:08,003] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:08,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:08,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:08,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:08,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,047] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,047] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,047] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,048] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:08,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:08,052] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,059] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,061] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:08,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:08,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:08,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:08,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:08,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 13: [2023-05-13 12:05:08,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,102] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:08,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 6: [2023-05-13 12:05:08,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,111] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,111] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 13: [2023-05-13 12:05:08,112] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,112] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 4: [2023-05-13 12:05:08,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 4: [2023-05-13 12:05:08,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 4: [2023-05-13 12:05:08,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:08,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,114] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 4: [2023-05-13 12:05:08,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:08,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 13: [2023-05-13 12:05:08,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt... 6: [2023-05-13 12:05:08,117] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:08,117] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:08,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:08,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:08,119] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,123] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,130] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,131] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,131] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,133] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,134] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:08,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:08,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:08,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:08,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:08,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:08,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:08,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:08,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:08,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:08,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,153] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,163] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,164] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 9: [2023-05-13 12:05:08,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 9: [2023-05-13 12:05:08,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,166] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,166] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,166] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,166] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,166] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,166] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 9: [2023-05-13 12:05:08,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:08,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:08,167] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,168] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 14: [2023-05-13 12:05:08,170] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,170] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,173] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,173] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,173] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:08,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:08,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,175] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:08,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:08,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:08,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:08,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:08,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:08,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:08,177] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:08,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:08,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:08,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 7: [2023-05-13 12:05:08,178] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,179] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,179] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 4: [2023-05-13 12:05:08,179] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_24-model_00-model_states.pt. 28: [2023-05-13 12:05:08,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:08,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:08,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:08,190] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:08,191] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:08,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 3: [2023-05-13 12:05:08,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 6: [2023-05-13 12:05:08,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 6: [2023-05-13 12:05:08,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,193] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:08,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:08,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,209] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 9: [2023-05-13 12:05:08,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 4: [2023-05-13 12:05:08,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 31: [2023-05-13 12:05:08,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 16: [2023-05-13 12:05:08,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 3: [2023-05-13 12:05:08,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,220] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,220] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,220] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,220] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,220] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,222] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,223] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 6: [2023-05-13 12:05:08,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 11: [2023-05-13 12:05:08,242] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:08,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 31: [2023-05-13 12:05:08,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 9: [2023-05-13 12:05:08,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:08,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:08,245] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 10: [2023-05-13 12:05:08,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 9: [2023-05-13 12:05:08,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:08,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,250] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:08,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,255] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 7: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 22: [2023-05-13 12:05:08,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 28: [2023-05-13 12:05:08,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:08,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:08,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:08,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:08,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 24: [2023-05-13 12:05:08,301] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:08,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 28: [2023-05-13 12:05:08,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,305] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:08,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:08,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:08,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:08,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:08,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:08,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 1: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:08,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:08,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:08,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:08,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:08,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:08,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,324] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:08,326] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,326] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,326] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:08,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:08,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:08,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:08,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:08,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:08,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 11: [2023-05-13 12:05:08,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,335] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,335] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 11: [2023-05-13 12:05:08,336] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,336] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:08,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:08,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:08,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,338] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,338] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 30: [2023-05-13 12:05:08,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:08,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:08,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:08,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:08,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:08,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:08,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:08,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:08,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:08,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:08,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:08,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:08,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 11: [2023-05-13 12:05:08,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:08,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:08,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:08,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:08,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:08,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 19: [2023-05-13 12:05:08,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:08,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:08,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 15: [2023-05-13 12:05:08,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 15: [2023-05-13 12:05:08,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,366] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:08,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:08,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,369] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:08,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 0: [2023-05-13 12:05:08,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,373] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,373] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,373] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,373] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:08,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:08,378] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 12: [2023-05-13 12:05:08,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:08,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 29: [2023-05-13 12:05:08,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:08,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:08,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 26: [2023-05-13 12:05:08,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 8: [2023-05-13 12:05:08,381] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,381] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,382] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 12: [2023-05-13 12:05:08,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,393] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,393] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 24: [2023-05-13 12:05:08,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,399] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:08,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:08,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 27: [2023-05-13 12:05:08,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:08,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:08,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:08,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:08,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:08,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:08,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:08,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:08,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 18: [2023-05-13 12:05:08,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 30: [2023-05-13 12:05:08,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:08,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 2: [2023-05-13 12:05:08,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:08,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 25: [2023-05-13 12:05:08,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 8: [2023-05-13 12:05:08,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,416] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,418] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 20: [2023-05-13 12:05:08,418] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,418] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:08,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 30: [2023-05-13 12:05:08,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:08,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:08,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:08,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:08,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:08,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 8: [2023-05-13 12:05:08,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,432] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 4: [2023-05-13 12:05:08,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 4: [2023-05-13 12:05:08,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 26: [2023-05-13 12:05:08,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 1: [2023-05-13 12:05:08,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:08,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 19: [2023-05-13 12:05:08,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 4: [2023-05-13 12:05:08,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 2: [2023-05-13 12:05:08,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt... 1: [2023-05-13 12:05:08,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 2: [2023-05-13 12:05:08,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,443] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,444] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 25: [2023-05-13 12:05:08,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,443] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 18: [2023-05-13 12:05:08,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 29: [2023-05-13 12:05:08,448] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 19: [2023-05-13 12:05:08,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,468] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,471] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 21: [2023-05-13 12:05:08,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,473] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 26: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,474] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,476] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,476] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,479] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 0: [2023-05-13 12:05:08,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 5: [2023-05-13 12:05:08,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 31: [2023-05-13 12:05:08,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:08,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:08,487] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:08,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:08,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 23: [2023-05-13 12:05:08,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:08,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:08,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 23: [2023-05-13 12:05:08,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:08,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:08,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 17: [2023-05-13 12:05:08,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 21: [2023-05-13 12:05:08,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 21: [2023-05-13 12:05:08,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:08,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 4: [2023-05-13 12:05:08,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_25-model_00-model_states.pt. 20: [2023-05-13 12:05:08,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,511] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:08,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:08,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 5: [2023-05-13 12:05:08,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,521] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:08,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,532] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 20: [2023-05-13 12:05:08,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 14: [2023-05-13 12:05:08,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:08,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 16: [2023-05-13 12:05:08,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 14: [2023-05-13 12:05:08,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 6: [2023-05-13 12:05:08,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 23: [2023-05-13 12:05:08,555] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,555] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,555] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:08,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:08,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:08,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:08,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 14: [2023-05-13 12:05:08,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,576] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:08,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,583] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:08,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:08,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 17: [2023-05-13 12:05:08,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 23: [2023-05-13 12:05:08,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:08,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 23: [2023-05-13 12:05:08,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 16: [2023-05-13 12:05:08,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:08,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:08,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,632] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,642] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,642] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,642] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,642] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:08,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,661] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,662] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:08,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:08,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:08,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:08,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,683] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:08,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:08,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 3: [2023-05-13 12:05:08,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 3: [2023-05-13 12:05:08,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:08,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:08,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 7: [2023-05-13 12:05:08,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:08,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 4: [2023-05-13 12:05:08,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt... 22: [2023-05-13 12:05:08,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 13: [2023-05-13 12:05:08,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:08,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 15: [2023-05-13 12:05:08,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:08,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:08,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 10: [2023-05-13 12:05:08,757] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,757] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,757] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:08,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:08,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 10: [2023-05-13 12:05:08,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:08,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:08,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:08,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:08,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 4: [2023-05-13 12:05:08,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_26-model_00-model_states.pt. 27: [2023-05-13 12:05:08,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:08,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:08,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 12: [2023-05-13 12:05:08,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 10: [2023-05-13 12:05:08,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:08,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:08,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:08,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 7: [2023-05-13 12:05:08,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:08,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 7: [2023-05-13 12:05:08,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 10: [2023-05-13 12:05:08,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 10: [2023-05-13 12:05:08,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 10: [2023-05-13 12:05:08,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:08,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:08,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 10: [2023-05-13 12:05:08,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,803] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 22: [2023-05-13 12:05:08,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:08,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:08,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:08,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:08,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:08,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:08,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:08,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:08,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:08,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:08,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:08,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:08,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 22: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:08,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 27: [2023-05-13 12:05:08,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:08,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:08,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 27: [2023-05-13 12:05:08,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:08,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:08,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:08,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:08,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:08,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:08,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:08,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:08,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,846] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,846] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 15: [2023-05-13 12:05:08,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:08,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:08,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,859] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,862] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:08,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:08,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:08,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 2: [2023-05-13 12:05:08,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:08,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 11: [2023-05-13 12:05:08,876] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,876] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 11: [2023-05-13 12:05:08,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,878] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:08,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:08,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 11: [2023-05-13 12:05:08,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 12: [2023-05-13 12:05:08,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:08,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:08,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 11: [2023-05-13 12:05:08,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 11: [2023-05-13 12:05:08,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 11: [2023-05-13 12:05:08,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 11: [2023-05-13 12:05:08,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,888] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 2: [2023-05-13 12:05:08,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:08,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:08,891] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:08,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:08,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:08,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:08,898] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:08,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 31: [2023-05-13 12:05:08,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:08,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 1: [2023-05-13 12:05:08,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 8: [2023-05-13 12:05:08,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:08,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 18: [2023-05-13 12:05:08,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 8: [2023-05-13 12:05:08,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:08,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:08,909] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,909] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,909] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,910] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,910] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,910] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:08,910] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,910] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:08,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:08,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 26: [2023-05-13 12:05:08,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:08,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:08,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,921] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:08,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:08,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:08,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 31: [2023-05-13 12:05:08,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:08,928] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 20: [2023-05-13 12:05:08,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 5: [2023-05-13 12:05:08,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 19: [2023-05-13 12:05:08,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:08,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:08,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:08,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 19: [2023-05-13 12:05:08,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:08,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 24: [2023-05-13 12:05:08,938] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 29: [2023-05-13 12:05:08,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,939] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:08,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:08,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:08,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:08,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:08,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 20: [2023-05-13 12:05:08,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:08,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:08,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:08,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:08,948] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,948] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,948] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,948] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:08,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:08,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:08,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:08,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:08,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:08,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:08,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:08,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 29: [2023-05-13 12:05:08,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 5: [2023-05-13 12:05:08,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:08,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,961] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:08,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:08,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:08,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:08,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,963] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:08,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:08,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:08,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 24: [2023-05-13 12:05:08,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,965] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:08,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:08,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:08,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:08,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:08,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,968] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:08,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:08,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:08,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:08,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 14: [2023-05-13 12:05:08,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:08,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:08,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:08,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:08,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:08,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 18: [2023-05-13 12:05:08,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 26: [2023-05-13 12:05:08,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 11: [2023-05-13 12:05:08,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:08,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:08,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:08,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:08,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 30: [2023-05-13 12:05:08,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 25: [2023-05-13 12:05:08,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:08,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,980] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:08,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:08,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:08,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 25: [2023-05-13 12:05:08,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:08,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:08,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:08,983] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 16: [2023-05-13 12:05:08,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:08,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:08,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 5: [2023-05-13 12:05:08,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:08,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:08,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:08,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 21: [2023-05-13 12:05:08,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:08,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:08,985] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:08,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 28: [2023-05-13 12:05:08,986] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:08,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:08,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:08,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:08,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:08,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:08,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:08,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:08,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:08,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:08,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 17: [2023-05-13 12:05:08,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 30: [2023-05-13 12:05:08,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:08,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:08,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:08,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:08,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:08,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:08,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:08,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:08,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:09,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:09,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:09,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:09,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,004] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:09,005] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:09,005] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:09,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:09,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:09,006] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:09,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 1: [2023-05-13 12:05:09,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:09,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:09,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:09,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:09,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:09,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:09,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 0: [2023-05-13 12:05:09,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 16: [2023-05-13 12:05:09,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:09,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:09,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:09,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:09,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:09,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:09,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,033] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:09,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:09,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:09,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:09,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:09,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:09,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:09,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 3: [2023-05-13 12:05:09,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:09,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:09,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:09,044] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,044] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:09,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:09,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:09,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:09,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,045] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 9: [2023-05-13 12:05:09,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:09,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:09,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:09,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:09,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 4: [2023-05-13 12:05:09,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 9: [2023-05-13 12:05:09,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:09,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:09,051] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt... 0: [2023-05-13 12:05:09,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:09,052] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:09,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:09,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:09,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,055] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:09,056] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:09,056] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:09,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:09,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:09,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:09,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 6: [2023-05-13 12:05:09,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:09,066] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:09,067] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:09,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:09,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:09,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,070] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 21: [2023-05-13 12:05:09,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:09,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,080] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:09,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:09,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:09,082] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,087] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,087] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,089] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,089] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,089] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,090] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,090] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,090] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,090] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,090] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:09,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:09,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:09,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:09,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,097] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:09,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 17: [2023-05-13 12:05:09,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:09,098] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 13: [2023-05-13 12:05:09,101] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:09,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:09,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,111] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,111] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 4: [2023-05-13 12:05:09,115] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_27-model_00-model_states.pt. 28: [2023-05-13 12:05:09,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,117] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,120] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,123] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,124] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:09,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:09,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:09,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:09,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:09,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:09,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:09,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:09,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 3: [2023-05-13 12:05:09,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,150] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,151] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,165] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,167] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,191] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,196] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,202] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 10: [2023-05-13 12:05:09,203] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:09,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,204] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 10: [2023-05-13 12:05:09,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 10: [2023-05-13 12:05:09,206] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,208] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:09,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:09,211] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,212] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:09,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 13: [2023-05-13 12:05:09,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:09,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:09,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,216] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,217] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:09,217] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,217] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 23: [2023-05-13 12:05:09,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 23: [2023-05-13 12:05:09,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,220] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,223] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,224] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:09,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,251] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 15: [2023-05-13 12:05:09,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,254] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 15: [2023-05-13 12:05:09,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,255] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:09,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:09,256] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:09,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:09,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:09,257] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:09,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,267] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,267] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,268] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,272] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,273] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:09,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,279] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 8: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,283] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,283] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,283] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:09,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:09,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:09,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:09,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:09,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:09,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:09,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:09,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:09,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:09,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 24: [2023-05-13 12:05:09,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:09,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:09,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:09,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:09,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 7: [2023-05-13 12:05:09,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 10: [2023-05-13 12:05:09,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:09,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:09,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:09,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:09,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 29: [2023-05-13 12:05:09,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:09,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:09,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:09,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:09,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:09,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 12: [2023-05-13 12:05:09,292] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:09,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 8: [2023-05-13 12:05:09,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:09,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:09,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 5: [2023-05-13 12:05:09,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 10: [2023-05-13 12:05:09,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:09,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 20: [2023-05-13 12:05:09,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 16: [2023-05-13 12:05:09,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,304] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:09,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:09,310] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,312] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,313] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,323] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,323] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,323] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,323] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,324] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,324] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,324] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,324] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,326] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,326] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,327] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:09,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:09,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,332] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,333] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:09,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 19: [2023-05-13 12:05:09,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:09,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:09,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,335] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:09,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 2: [2023-05-13 12:05:09,338] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 19: [2023-05-13 12:05:09,339] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,338] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,338] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:09,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:09,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:09,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:09,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,342] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 7: [2023-05-13 12:05:09,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 22: [2023-05-13 12:05:09,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 1: [2023-05-13 12:05:09,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 30: [2023-05-13 12:05:09,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 27: [2023-05-13 12:05:09,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 27: [2023-05-13 12:05:09,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,351] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 29: [2023-05-13 12:05:09,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 2: [2023-05-13 12:05:09,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,359] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 31: [2023-05-13 12:05:09,361] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:09,361] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,361] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 31: [2023-05-13 12:05:09,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:09,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:09,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:09,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:09,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,366] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,367] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,367] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 24: [2023-05-13 12:05:09,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,368] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,369] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,369] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:09,373] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,374] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 5: [2023-05-13 12:05:09,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,377] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,379] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 12: [2023-05-13 12:05:09,381] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,382] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,382] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 22: [2023-05-13 12:05:09,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 5: [2023-05-13 12:05:09,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:09,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,391] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,391] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,393] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,395] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,396] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 20: [2023-05-13 12:05:09,398] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,399] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:09,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 11: [2023-05-13 12:05:09,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 5: [2023-05-13 12:05:09,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,404] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:09,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:09,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 17: [2023-05-13 12:05:09,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:09,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 25: [2023-05-13 12:05:09,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 26: [2023-05-13 12:05:09,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 5: [2023-05-13 12:05:09,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 9: [2023-05-13 12:05:09,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 6: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 16: [2023-05-13 12:05:09,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 9: [2023-05-13 12:05:09,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 0: [2023-05-13 12:05:09,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:09,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:09,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 14: [2023-05-13 12:05:09,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,418] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 21: [2023-05-13 12:05:09,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 20: [2023-05-13 12:05:09,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 6: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 20: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 30: [2023-05-13 12:05:09,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,427] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 26: [2023-05-13 12:05:09,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,430] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 28: [2023-05-13 12:05:09,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 11: [2023-05-13 12:05:09,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 18: [2023-05-13 12:05:09,437] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,437] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,437] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,437] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 4: [2023-05-13 12:05:09,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt... 25: [2023-05-13 12:05:09,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:09,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:09,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,448] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 13: [2023-05-13 12:05:09,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 3: [2023-05-13 12:05:09,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 3: [2023-05-13 12:05:09,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 18: [2023-05-13 12:05:09,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:09,473] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 3: [2023-05-13 12:05:09,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 1: [2023-05-13 12:05:09,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 28: [2023-05-13 12:05:09,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 0: [2023-05-13 12:05:09,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:09,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,493] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,494] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 14: [2023-05-13 12:05:09,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 17: [2023-05-13 12:05:09,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 21: [2023-05-13 12:05:09,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_28-model_00-model_states.pt. 4: [2023-05-13 12:05:09,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,525] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,525] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,525] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,525] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,533] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 28: [2023-05-13 12:05:09,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,570] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:09,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,584] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:09,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,606] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 10: [2023-05-13 12:05:09,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:09,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,637] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:09,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 10: [2023-05-13 12:05:09,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:09,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,641] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:09,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:09,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:09,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:09,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:09,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:09,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:09,654] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:09,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 12: [2023-05-13 12:05:09,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,662] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,662] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,662] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:09,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,663] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:09,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,664] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:09,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:09,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 24: [2023-05-13 12:05:09,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:09,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:09,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 15: [2023-05-13 12:05:09,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:09,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 7: [2023-05-13 12:05:09,680] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 7: [2023-05-13 12:05:09,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:09,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:09,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:09,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:09,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:09,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 29: [2023-05-13 12:05:09,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:09,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 8: [2023-05-13 12:05:09,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:09,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,709] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,711] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 8: [2023-05-13 12:05:09,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:09,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:09,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 15: [2023-05-13 12:05:09,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:09,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 16: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:09,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,723] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:09,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:09,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:09,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:09,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 20: [2023-05-13 12:05:09,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:09,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 20: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 20: [2023-05-13 12:05:09,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 24: [2023-05-13 12:05:09,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 27: [2023-05-13 12:05:09,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 22: [2023-05-13 12:05:09,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:09,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 31: [2023-05-13 12:05:09,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:09,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:09,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:09,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:09,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 19: [2023-05-13 12:05:09,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 29: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:09,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,744] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:09,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:09,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:09,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:09,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:09,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:09,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:09,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,746] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:09,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 22: [2023-05-13 12:05:09,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 2: [2023-05-13 12:05:09,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,753] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,753] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 6: [2023-05-13 12:05:09,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:09,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:09,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:09,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:09,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:09,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:09,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 17: [2023-05-13 12:05:09,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,757] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 12: [2023-05-13 12:05:09,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:09,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:09,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 13: [2023-05-13 12:05:09,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:09,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 27: [2023-05-13 12:05:09,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:09,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:09,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:09,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 13: [2023-05-13 12:05:09,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 13: [2023-05-13 12:05:09,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 13: [2023-05-13 12:05:09,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 13: [2023-05-13 12:05:09,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 13: [2023-05-13 12:05:09,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:09,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:09,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:09,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:09,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:09,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 16: [2023-05-13 12:05:09,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:09,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 2: [2023-05-13 12:05:09,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:09,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:09,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:09,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:09,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:09,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:09,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 20: [2023-05-13 12:05:09,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:09,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,768] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:09,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:09,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:09,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:09,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:09,771] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:09,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:09,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,772] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:09,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:09,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:09,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,773] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,774] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:09,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:09,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 30: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:09,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 5: [2023-05-13 12:05:09,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:09,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 26: [2023-05-13 12:05:09,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:09,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:09,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 5: [2023-05-13 12:05:09,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 5: [2023-05-13 12:05:09,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 30: [2023-05-13 12:05:09,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,782] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:09,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 25: [2023-05-13 12:05:09,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,785] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 9: [2023-05-13 12:05:09,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:09,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 20: [2023-05-13 12:05:09,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:09,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 18: [2023-05-13 12:05:09,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 18: [2023-05-13 12:05:09,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 21: [2023-05-13 12:05:09,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 19: [2023-05-13 12:05:09,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 1: [2023-05-13 12:05:09,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:09,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,794] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 31: [2023-05-13 12:05:09,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:09,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:09,795] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:09,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:09,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:09,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:09,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:09,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:09,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,800] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 23: [2023-05-13 12:05:09,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:09,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,802] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:09,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 26: [2023-05-13 12:05:09,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 23: [2023-05-13 12:05:09,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:09,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:09,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:09,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 0: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 11: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:09,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 13: [2023-05-13 12:05:09,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 13: [2023-05-13 12:05:09,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:09,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:09,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:09,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:09,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:09,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 14: [2023-05-13 12:05:09,820] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:09,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 4: [2023-05-13 12:05:09,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt... 9: [2023-05-13 12:05:09,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:09,822] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:09,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 9: [2023-05-13 12:05:09,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:09,827] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,827] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:09,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:09,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:09,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:09,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:09,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:09,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,831] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 28: [2023-05-13 12:05:09,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:09,836] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:09,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 13: [2023-05-13 12:05:09,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 13: [2023-05-13 12:05:09,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:09,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:09,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:09,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:09,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:09,842] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:09,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:09,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:09,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:09,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:09,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 25: [2023-05-13 12:05:09,846] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:09,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,848] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:09,850] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:09,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:09,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 11: [2023-05-13 12:05:09,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:09,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 20: [2023-05-13 12:05:09,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:09,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:09,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 20: [2023-05-13 12:05:09,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:09,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:09,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:09,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,856] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:09,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:09,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:09,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:09,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,860] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 6: [2023-05-13 12:05:09,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:09,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:09,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:09,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:09,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:09,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:09,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:09,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:09,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:09,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:09,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:09,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:09,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:09,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 17: [2023-05-13 12:05:09,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,871] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:09,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:09,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 13: [2023-05-13 12:05:09,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:09,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 14: [2023-05-13 12:05:09,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:09,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:09,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:09,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:09,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:09,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:09,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:09,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:09,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 5: [2023-05-13 12:05:09,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 21: [2023-05-13 12:05:09,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 0: [2023-05-13 12:05:09,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 13: [2023-05-13 12:05:09,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:09,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:09,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:09,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 1: [2023-05-13 12:05:09,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_29-model_00-model_states.pt. 4: [2023-05-13 12:05:09,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:09,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:09,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:09,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:09,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 28: [2023-05-13 12:05:09,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:09,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:09,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:09,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:09,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:09,927] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:09,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:09,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:09,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:09,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:09,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:09,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:09,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:09,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:09,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:09,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:09,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:09,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:09,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:09,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:09,969] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:09,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:09,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:09,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:09,978] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:09,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:09,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:09,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:09,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:09,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:09,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:09,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:09,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:09,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:09,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:09,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,000] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,001] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:10,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:10,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:10,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:10,005] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:10,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:10,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:10,009] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:10,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:10,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:10,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:10,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,010] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,011] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:10,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:10,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 10: [2023-05-13 12:05:10,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,015] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:10,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,017] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 3: [2023-05-13 12:05:10,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:10,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:10,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:10,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:10,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:10,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:10,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:10,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,034] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,035] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 3: [2023-05-13 12:05:10,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,036] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:10,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:10,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,039] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:10,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:10,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:10,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:10,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 10: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,046] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:10,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,048] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:10,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:10,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:10,049] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,052] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:10,053] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:10,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:10,054] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:10,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:10,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 7: [2023-05-13 12:05:10,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:10,058] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:10,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,059] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 10: [2023-05-13 12:05:10,060] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 10: [2023-05-13 12:05:10,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,064] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:10,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:10,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:10,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 15: [2023-05-13 12:05:10,066] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:10,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:10,071] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 29: [2023-05-13 12:05:10,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 8: [2023-05-13 12:05:10,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,076] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,077] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 29: [2023-05-13 12:05:10,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,077] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,077] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,077] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:10,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,078] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,079] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:10,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:10,081] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,082] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,083] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 12: [2023-05-13 12:05:10,083] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,084] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:10,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,084] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,085] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,087] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,088] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,089] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,092] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:10,092] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,094] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,098] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,100] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,101] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 15: [2023-05-13 12:05:10,105] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,107] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,109] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 7: [2023-05-13 12:05:10,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,111] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,113] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 8: [2023-05-13 12:05:10,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,113] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,114] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,120] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:10,123] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,123] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,123] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,124] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:10,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 31: [2023-05-13 12:05:10,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,127] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 19: [2023-05-13 12:05:10,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:10,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:10,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 27: [2023-05-13 12:05:10,129] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 2: [2023-05-13 12:05:10,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 22: [2023-05-13 12:05:10,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,134] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:10,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,134] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,134] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:10,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,136] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,136] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,136] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,136] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:10,136] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,136] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,138] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,138] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 12: [2023-05-13 12:05:10,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:10,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,141] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,141] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,142] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:10,142] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:10,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,142] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:10,143] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 19: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 2: [2023-05-13 12:05:10,144] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:10,147] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:10,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:10,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:10,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:10,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:10,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:10,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:10,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 22: [2023-05-13 12:05:10,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:10,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:10,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:10,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:10,150] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:10,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:10,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:10,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 18: [2023-05-13 12:05:10,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:10,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,153] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,155] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,156] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 0: [2023-05-13 12:05:10,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 27: [2023-05-13 12:05:10,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,158] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 23: [2023-05-13 12:05:10,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,161] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,162] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,165] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 23: [2023-05-13 12:05:10,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,166] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,166] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 16: [2023-05-13 12:05:10,168] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:10,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:10,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,176] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:10,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:10,179] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,179] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,179] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,179] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,179] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:10,179] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:10,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,182] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:10,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:10,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:10,184] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:10,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 16: [2023-05-13 12:05:10,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:10,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:10,185] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:10,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:10,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 11: [2023-05-13 12:05:10,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:10,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:10,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 14: [2023-05-13 12:05:10,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:10,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 21: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:10,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,188] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:10,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,189] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 25: [2023-05-13 12:05:10,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 6: [2023-05-13 12:05:10,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,190] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,190] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,190] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,190] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,190] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:10,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:10,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:10,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:10,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:10,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:10,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:10,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:10,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:10,195] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:10,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:10,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:10,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:10,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:10,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 5: [2023-05-13 12:05:10,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 30: [2023-05-13 12:05:10,198] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,198] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 17: [2023-05-13 12:05:10,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 26: [2023-05-13 12:05:10,198] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:10,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:10,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:10,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 1: [2023-05-13 12:05:10,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 4: [2023-05-13 12:05:10,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt... 24: [2023-05-13 12:05:10,200] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 5: [2023-05-13 12:05:10,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,201] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,204] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,205] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,205] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:10,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:10,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,209] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,209] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,210] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 30: [2023-05-13 12:05:10,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,215] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,215] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 24: [2023-05-13 12:05:10,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 26: [2023-05-13 12:05:10,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,222] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:10,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 9: [2023-05-13 12:05:10,226] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 9: [2023-05-13 12:05:10,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,230] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:10,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:10,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:10,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,235] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,236] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 9: [2023-05-13 12:05:10,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 9: [2023-05-13 12:05:10,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 9: [2023-05-13 12:05:10,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,239] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 9: [2023-05-13 12:05:10,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:10,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:10,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 9: [2023-05-13 12:05:10,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 20: [2023-05-13 12:05:10,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:10,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,244] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:10,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,246] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,246] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:10,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,247] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 31: [2023-05-13 12:05:10,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,248] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 25: [2023-05-13 12:05:10,249] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 18: [2023-05-13 12:05:10,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:10,252] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,253] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:10,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,256] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 6: [2023-05-13 12:05:10,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,260] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:10,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 4: [2023-05-13 12:05:10,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,271] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 14: [2023-05-13 12:05:10,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 11: [2023-05-13 12:05:10,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 0: [2023-05-13 12:05:10,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,279] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,280] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,281] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 9: [2023-05-13 12:05:10,283] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,284] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:10,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 1: [2023-05-13 12:05:10,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 21: [2023-05-13 12:05:10,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,293] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 17: [2023-05-13 12:05:10,298] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_30-model_00-model_states.pt. 28: [2023-05-13 12:05:10,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,299] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,308] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 9: [2023-05-13 12:05:10,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 13: [2023-05-13 12:05:10,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:10,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,326] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 9: [2023-05-13 12:05:10,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,328] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,329] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,330] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 9: [2023-05-13 12:05:10,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,359] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,360] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,361] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 10: [2023-05-13 12:05:10,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 10: [2023-05-13 12:05:10,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 10: [2023-05-13 12:05:10,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 10: [2023-05-13 12:05:10,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 28: [2023-05-13 12:05:10,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,401] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 10: [2023-05-13 12:05:10,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,424] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,433] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,443] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,453] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,455] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,455] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 10: [2023-05-13 12:05:10,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 10: [2023-05-13 12:05:10,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,465] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,467] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 10: [2023-05-13 12:05:10,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,473] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,473] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 2: [2023-05-13 12:05:10,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,487] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,494] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 2: [2023-05-13 12:05:10,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,506] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,516] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 3: [2023-05-13 12:05:10,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 3: [2023-05-13 12:05:10,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 21: [2023-05-13 12:05:10,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,529] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 16: [2023-05-13 12:05:10,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 29: [2023-05-13 12:05:10,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 16: [2023-05-13 12:05:10,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 16: [2023-05-13 12:05:10,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 16: [2023-05-13 12:05:10,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 12: [2023-05-13 12:05:10,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,543] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 16: [2023-05-13 12:05:10,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 16: [2023-05-13 12:05:10,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 31: [2023-05-13 12:05:10,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,557] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,558] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 27: [2023-05-13 12:05:10,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 27: [2023-05-13 12:05:10,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,576] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,576] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,576] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,579] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 12: [2023-05-13 12:05:10,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,584] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,583] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,587] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,589] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 20: [2023-05-13 12:05:10,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,590] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 23: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 27: [2023-05-13 12:05:10,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 26: [2023-05-13 12:05:10,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 26: [2023-05-13 12:05:10,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 27: [2023-05-13 12:05:10,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 27: [2023-05-13 12:05:10,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 24: [2023-05-13 12:05:10,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 27: [2023-05-13 12:05:10,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 27: [2023-05-13 12:05:10,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 16: [2023-05-13 12:05:10,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 16: [2023-05-13 12:05:10,601] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,604] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,605] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 11: [2023-05-13 12:05:10,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 5: [2023-05-13 12:05:10,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 22: [2023-05-13 12:05:10,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 21: [2023-05-13 12:05:10,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,609] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,610] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,610] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,610] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,612] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 7: [2023-05-13 12:05:10,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 6: [2023-05-13 12:05:10,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 16: [2023-05-13 12:05:10,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 6: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 6: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 6: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 6: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 7: [2023-05-13 12:05:10,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 4: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 18: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 6: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 31: [2023-05-13 12:05:10,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 17: [2023-05-13 12:05:10,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 8: [2023-05-13 12:05:10,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,624] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 16: [2023-05-13 12:05:10,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:10,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 5: [2023-05-13 12:05:10,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:10,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:10,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 30: [2023-05-13 12:05:10,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 0: [2023-05-13 12:05:10,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,632] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 30: [2023-05-13 12:05:10,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 22: [2023-05-13 12:05:10,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 25: [2023-05-13 12:05:10,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 19: [2023-05-13 12:05:10,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 16: [2023-05-13 12:05:10,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,643] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,643] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,644] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 16: [2023-05-13 12:05:10,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:10,649] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:10,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:10,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,657] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,658] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 23: [2023-05-13 12:05:10,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,660] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,662] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,667] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,668] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 20: [2023-05-13 12:05:10,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,673] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 13: [2023-05-13 12:05:10,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 14: [2023-05-13 12:05:10,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,680] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 4: [2023-05-13 12:05:10,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,686] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:10,687] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:10,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:10,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 24: [2023-05-13 12:05:10,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,691] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,692] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,693] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 31: [2023-05-13 12:05:10,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:10,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:10,661] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:10,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:10,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:10,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:10,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:10,689] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:10,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 6: [2023-05-13 12:05:10,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 11: [2023-05-13 12:05:10,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:10,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 6: [2023-05-13 12:05:10,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 6: [2023-05-13 12:05:10,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 6: [2023-05-13 12:05:10,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,702] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 25: [2023-05-13 12:05:10,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 19: [2023-05-13 12:05:10,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 0: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 28: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:10,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,708] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:10,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:10,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:10,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:10,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:10,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:10,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:10,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:10,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 17: [2023-05-13 12:05:10,718] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 18: [2023-05-13 12:05:10,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:10,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:10,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:10,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:10,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:10,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 6: [2023-05-13 12:05:10,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:10,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:10,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:10,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:10,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,744] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:10,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:10,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:10,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:10,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 28: [2023-05-13 12:05:10,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:10,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:10,746] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:10,747] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,755] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 14: [2023-05-13 12:05:10,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:10,756] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,759] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,760] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,767] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 29: [2023-05-13 12:05:10,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 13: [2023-05-13 12:05:10,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:10,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:10,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 15: [2023-05-13 12:05:10,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:10,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 14: [2023-05-13 12:05:10,777] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:10,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:10,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:10,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,780] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:10,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 1: [2023-05-13 12:05:10,784] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt... 14: [2023-05-13 12:05:10,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:10,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:10,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:10,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:10,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 13: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,797] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 12: [2023-05-13 12:05:10,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,800] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,800] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,804] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,813] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:10,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:10,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:10,822] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,822] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,822] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:10,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:10,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:10,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:10,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 10: [2023-05-13 12:05:10,835] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,837] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,837] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:10,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 8: [2023-05-13 12:05:10,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 10: [2023-05-13 12:05:10,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:10,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:10,845] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,845] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,845] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:10,844] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 29: [2023-05-13 12:05:10,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,848] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,849] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:10,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 3: [2023-05-13 12:05:10,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,851] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 15: [2023-05-13 12:05:10,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 3: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:10,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:10,854] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:10,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:10,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:10,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:10,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:10,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:10,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 2: [2023-05-13 12:05:10,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:10,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:10,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,859] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 2: [2023-05-13 12:05:10,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:10,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 29: [2023-05-13 12:05:10,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:10,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:10,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:10,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:10,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:10,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:10,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 1: [2023-05-13 12:05:10,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_31-model_00-model_states.pt. 15: [2023-05-13 12:05:10,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:10,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:10,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:10,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:10,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:10,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,883] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 15: [2023-05-13 12:05:10,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:10,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:10,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:10,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:10,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:10,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:10,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,892] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,892] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,892] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,893] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 9: [2023-05-13 12:05:10,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:10,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:10,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:10,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:10,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 27: [2023-05-13 12:05:10,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 9: [2023-05-13 12:05:10,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 27: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 27: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 12: [2023-05-13 12:05:10,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:10,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 7: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,906] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:10,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,909] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,909] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:10,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 30: [2023-05-13 12:05:10,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:10,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 7: [2023-05-13 12:05:10,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:10,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:10,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 8: [2023-05-13 12:05:10,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:10,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 21: [2023-05-13 12:05:10,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:10,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 21: [2023-05-13 12:05:10,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:10,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 8: [2023-05-13 12:05:10,929] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:10,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:10,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:10,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:10,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:10,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:10,945] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:10,945] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,945] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:10,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:10,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:10,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:10,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 5: [2023-05-13 12:05:10,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:10,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:10,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:10,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 18: [2023-05-13 12:05:10,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 5: [2023-05-13 12:05:10,954] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,954] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 30: [2023-05-13 12:05:10,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:10,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,957] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,959] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,959] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,964] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,965] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,966] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 26: [2023-05-13 12:05:10,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,968] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,970] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 23: [2023-05-13 12:05:10,970] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 19: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,971] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 11: [2023-05-13 12:05:10,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:10,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,973] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:10,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 4: [2023-05-13 12:05:10,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 0: [2023-05-13 12:05:10,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,976] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:10,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 22: [2023-05-13 12:05:10,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 22: [2023-05-13 12:05:10,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 25: [2023-05-13 12:05:10,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:10,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:10,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 23: [2023-05-13 12:05:10,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:10,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 26: [2023-05-13 12:05:10,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:10,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:10,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:10,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:10,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 27: [2023-05-13 12:05:10,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:10,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:10,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:10,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:10,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:10,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:10,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:10,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:10,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:10,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:10,988] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:10,989] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:10,990] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:10,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:10,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:10,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:10,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:10,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:10,993] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:10,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:10,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:10,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:10,996] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:10,997] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:10,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,974] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,979] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,996] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:10,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:10,997] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:10,996] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:10,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:10,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:10,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:10,998] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 24: [2023-05-13 12:05:10,999] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:10,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:10,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:10,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:10,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:10,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:10,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:10,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:10,999] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:10,961] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 11: [2023-05-13 12:05:11,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:11,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:10,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,974] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 17: [2023-05-13 12:05:10,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 24: [2023-05-13 12:05:11,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,005] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:11,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:11,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:11,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,008] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,010] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:11,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,013] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:11,014] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:11,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:11,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:11,023] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 18: [2023-05-13 12:05:11,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:11,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:11,024] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 19: [2023-05-13 12:05:11,024] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:11,029] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:11,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,033] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,034] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:11,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:11,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:11,037] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:11,037] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,038] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:11,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:11,039] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:11,040] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:11,041] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:11,041] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:11,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 20: [2023-05-13 12:05:11,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:11,043] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:11,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:11,044] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:11,045] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:11,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:11,046] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,049] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:11,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,051] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,050] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:11,051] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:11,053] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:11,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:11,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:11,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:11,057] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 4: [2023-05-13 12:05:11,062] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,063] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,064] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:11,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:11,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:11,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 17: [2023-05-13 12:05:11,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 25: [2023-05-13 12:05:11,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,066] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,069] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,072] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,079] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,081] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,086] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:11,087] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,090] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,090] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,090] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,090] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 0: [2023-05-13 12:05:11,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,093] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,095] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,096] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,099] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,100] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,114] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,116] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,118] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,120] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,120] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,121] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,126] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:11,128] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,130] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 20: [2023-05-13 12:05:11,131] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,131] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,132] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,102] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:11,103] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:11,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:11,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:11,104] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:11,105] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:11,106] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 1: [2023-05-13 12:05:11,107] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt... 20: [2023-05-13 12:05:11,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,168] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,171] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,172] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 20: [2023-05-13 12:05:11,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,172] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,177] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,179] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:11,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,180] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 28: [2023-05-13 12:05:11,181] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:11,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:11,182] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:11,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,183] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:11,187] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,186] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,200] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,201] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,203] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 1: [2023-05-13 12:05:11,208] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_32-model_00-model_states.pt. 31: [2023-05-13 12:05:11,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,214] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,214] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:11,216] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,218] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,219] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,220] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,220] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,220] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,220] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:11,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,223] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:11,223] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,224] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:11,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,227] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,227] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 14: [2023-05-13 12:05:11,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,228] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 14: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 14: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 14: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,229] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 2: [2023-05-13 12:05:11,230] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,231] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:11,232] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,234] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 6: [2023-05-13 12:05:11,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:11,236] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:11,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,237] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:11,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:11,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,238] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,239] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 6: [2023-05-13 12:05:11,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,240] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,242] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,243] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,244] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,251] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,263] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,265] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:11,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:11,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:11,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:11,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:11,272] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:11,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:11,273] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:11,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:11,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,274] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,274] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 3: [2023-05-13 12:05:11,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:11,275] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:11,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,276] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:11,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,277] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:11,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,278] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,285] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,287] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,288] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 14: [2023-05-13 12:05:11,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,295] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,299] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:11,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:11,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 14: [2023-05-13 12:05:11,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,303] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:11,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:11,306] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,307] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,309] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,309] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 28: [2023-05-13 12:05:11,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,310] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 14: [2023-05-13 12:05:11,311] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,311] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 14: [2023-05-13 12:05:11,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,314] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:11,315] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:11,316] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:11,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,317] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 5: [2023-05-13 12:05:11,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 14: [2023-05-13 12:05:11,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,318] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,319] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,320] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,322] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,323] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,325] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,326] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,327] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,329] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,331] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,344] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,345] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 10: [2023-05-13 12:05:11,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,347] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 5: [2023-05-13 12:05:11,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:11,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:11,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:11,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 10: [2023-05-13 12:05:11,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:11,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:11,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:11,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:11,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:11,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:11,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:11,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:11,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:11,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 15: [2023-05-13 12:05:11,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:11,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:11,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:11,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:11,353] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 14: [2023-05-13 12:05:11,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:11,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 9: [2023-05-13 12:05:11,354] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,355] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 3: [2023-05-13 12:05:11,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,358] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,363] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 15: [2023-05-13 12:05:11,366] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,368] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,375] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,376] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,384] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 21: [2023-05-13 12:05:11,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,392] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,394] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,394] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,399] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,399] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,401] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:11,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:11,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:11,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:11,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:11,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:11,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:11,408] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:11,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:11,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 21: [2023-05-13 12:05:11,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 7: [2023-05-13 12:05:11,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:11,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:11,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,414] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,414] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:11,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 29: [2023-05-13 12:05:11,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 29: [2023-05-13 12:05:11,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 13: [2023-05-13 12:05:11,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 12: [2023-05-13 12:05:11,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,418] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:11,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 23: [2023-05-13 12:05:11,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:11,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:11,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:11,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:11,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:11,420] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:11,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 23: [2023-05-13 12:05:11,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 13: [2023-05-13 12:05:11,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 12: [2023-05-13 12:05:11,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,428] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,431] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,435] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,444] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,453] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 11: [2023-05-13 12:05:11,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:11,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,475] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,479] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,479] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:11,479] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 26: [2023-05-13 12:05:11,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,480] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,481] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 0: [2023-05-13 12:05:11,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:11,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,487] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:11,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:11,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:11,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,490] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,492] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,492] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,494] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:11,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 11: [2023-05-13 12:05:11,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 27: [2023-05-13 12:05:11,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:11,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:11,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:11,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,505] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,506] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,512] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 26: [2023-05-13 12:05:11,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 27: [2023-05-13 12:05:11,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,509] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 25: [2023-05-13 12:05:11,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,518] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,519] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 18: [2023-05-13 12:05:11,520] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 0: [2023-05-13 12:05:11,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,525] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,526] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,528] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:11,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:11,529] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:11,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:11,530] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:11,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:11,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 30: [2023-05-13 12:05:11,531] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 19: [2023-05-13 12:05:11,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 24: [2023-05-13 12:05:11,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:11,532] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 17: [2023-05-13 12:05:11,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:11,534] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:11,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:11,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:11,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:11,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 8: [2023-05-13 12:05:11,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 17: [2023-05-13 12:05:11,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,541] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,541] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 16: [2023-05-13 12:05:11,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,550] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 22: [2023-05-13 12:05:11,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,551] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,551] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 1: [2023-05-13 12:05:11,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 31: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt... 4: [2023-05-13 12:05:11,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,556] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,558] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,560] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 16: [2023-05-13 12:05:11,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,562] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 31: [2023-05-13 12:05:11,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,574] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 4: [2023-05-13 12:05:11,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:11,574] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,575] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:11,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,577] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,578] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,579] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 16: [2023-05-13 12:05:11,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:11,585] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:11,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 18: [2023-05-13 12:05:11,588] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 25: [2023-05-13 12:05:11,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 24: [2023-05-13 12:05:11,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 16: [2023-05-13 12:05:11,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:11,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 30: [2023-05-13 12:05:11,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 19: [2023-05-13 12:05:11,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,599] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,599] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 9: [2023-05-13 12:05:11,600] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,602] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 22: [2023-05-13 12:05:11,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,607] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,607] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,609] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,610] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,612] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,612] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,612] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,613] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,616] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,623] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,624] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,625] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,625] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,625] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,626] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,627] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,629] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,630] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 2: [2023-05-13 12:05:11,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,632] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 28: [2023-05-13 12:05:11,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 8: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,635] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,636] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,644] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,645] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,645] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,648] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,649] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 7: [2023-05-13 12:05:11,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,651] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,652] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,652] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,653] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,655] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,655] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,656] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,659] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,662] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,663] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_33-model_00-model_states.pt. 1: [2023-05-13 12:05:11,666] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,667] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,668] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 31: [2023-05-13 12:05:11,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,674] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:11,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,675] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,678] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,679] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,681] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:11,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,685] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:11,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,686] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,688] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,688] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,689] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,692] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,693] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:11,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 5: [2023-05-13 12:05:11,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:11,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 10: [2023-05-13 12:05:11,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 5: [2023-05-13 12:05:11,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 31: [2023-05-13 12:05:11,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 5: [2023-05-13 12:05:11,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:11,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:11,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,701] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:11,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,710] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,712] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:11,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 6: [2023-05-13 12:05:11,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,716] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:11,717] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:11,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,717] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:11,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:11,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 3: [2023-05-13 12:05:11,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:11,720] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,721] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 10: [2023-05-13 12:05:11,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:11,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:11,722] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,723] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,723] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,723] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,724] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 14: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 21: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 15: [2023-05-13 12:05:11,727] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:11,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 9: [2023-05-13 12:05:11,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:11,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:11,730] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:11,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:11,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:11,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:11,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 2: [2023-05-13 12:05:11,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 5: [2023-05-13 12:05:11,731] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 9: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:11,732] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,733] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 2: [2023-05-13 12:05:11,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:11,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:11,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 28: [2023-05-13 12:05:11,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:11,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 5: [2023-05-13 12:05:11,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:11,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 5: [2023-05-13 12:05:11,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:11,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:11,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:11,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,740] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:11,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 27: [2023-05-13 12:05:11,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 20: [2023-05-13 12:05:11,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:11,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:11,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:11,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 20: [2023-05-13 12:05:11,745] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,745] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,747] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 3: [2023-05-13 12:05:11,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 0: [2023-05-13 12:05:11,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 28: [2023-05-13 12:05:11,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:11,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:11,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 6: [2023-05-13 12:05:11,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:11,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:11,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:11,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:11,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:11,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:11,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:11,753] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:11,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:11,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:11,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:11,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 21: [2023-05-13 12:05:11,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,757] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,758] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,758] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 17: [2023-05-13 12:05:11,759] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 23: [2023-05-13 12:05:11,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,760] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:11,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:11,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:11,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,761] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 15: [2023-05-13 12:05:11,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 23: [2023-05-13 12:05:11,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:11,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:11,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:11,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:11,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:11,766] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:11,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:11,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 7: [2023-05-13 12:05:11,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:11,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 13: [2023-05-13 12:05:11,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 7: [2023-05-13 12:05:11,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:11,776] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:11,778] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,781] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,783] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,783] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,788] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,798] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,798] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,800] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,802] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,810] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,812] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:11,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:11,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 29: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:11,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 13: [2023-05-13 12:05:11,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:11,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:11,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:11,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 12: [2023-05-13 12:05:11,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:11,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:11,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:11,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 29: [2023-05-13 12:05:11,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:11,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:11,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:11,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:11,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:11,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:11,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:11,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 27: [2023-05-13 12:05:11,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,835] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,835] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 12: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:11,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 14: [2023-05-13 12:05:11,842] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:11,849] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,856] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:11,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,862] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:11,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:11,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:11,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:11,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 0: [2023-05-13 12:05:11,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:11,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 26: [2023-05-13 12:05:11,867] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 26: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,872] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,872] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,872] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,876] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 22: [2023-05-13 12:05:11,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 8: [2023-05-13 12:05:11,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,879] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 18: [2023-05-13 12:05:11,880] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 25: [2023-05-13 12:05:11,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,888] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,889] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,889] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 24: [2023-05-13 12:05:11,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 4: [2023-05-13 12:05:11,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,894] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 11: [2023-05-13 12:05:11,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:11,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:11,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:11,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:11,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,897] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 17: [2023-05-13 12:05:11,878] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:11,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,901] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,903] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:11,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:11,905] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:11,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:11,907] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:11,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:11,908] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 19: [2023-05-13 12:05:11,911] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:11,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 1: [2023-05-13 12:05:11,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 25: [2023-05-13 12:05:11,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:11,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:11,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 11: [2023-05-13 12:05:11,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:11,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:11,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt... 30: [2023-05-13 12:05:11,888] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,911] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:11,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:11,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:11,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:11,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:11,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:11,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:11,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:11,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:11,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:11,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:11,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:11,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:11,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,924] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:11,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 4: [2023-05-13 12:05:11,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,928] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,929] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 30: [2023-05-13 12:05:11,931] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:11,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:11,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,937] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 19: [2023-05-13 12:05:11,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,948] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,949] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,949] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,950] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,950] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,951] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,951] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 22: [2023-05-13 12:05:11,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:11,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 24: [2023-05-13 12:05:11,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 8: [2023-05-13 12:05:11,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:11,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:11,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,967] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,969] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:11,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:11,971] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,976] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,977] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:11,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:11,978] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:11,979] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:11,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:11,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:11,982] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:11,982] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 18: [2023-05-13 12:05:11,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:11,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:11,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:11,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:11,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:11,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:12,008] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:12,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:12,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:12,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:12,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_34-model_00-model_states.pt. 1: [2023-05-13 12:05:12,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,028] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,029] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,030] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,042] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,043] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,047] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,050] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,054] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,055] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,056] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,057] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,062] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,065] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,068] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,069] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,070] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,071] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,073] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,074] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,076] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,091] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,096] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,099] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,102] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,103] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,104] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,106] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,108] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,110] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,109] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,115] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,116] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,117] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,118] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,123] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,125] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,125] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,127] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,128] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 5: [2023-05-13 12:05:12,129] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 5: [2023-05-13 12:05:12,133] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 5: [2023-05-13 12:05:12,135] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,137] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,137] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,138] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 5: [2023-05-13 12:05:12,138] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,139] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,139] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,140] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,144] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,144] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,145] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,145] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,146] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,147] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,148] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,148] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,149] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,149] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,151] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,152] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,154] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,157] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:12,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:12,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,157] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,159] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:12,160] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,163] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,164] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 16: [2023-05-13 12:05:12,166] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,169] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,171] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,173] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,173] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,173] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,174] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,174] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,176] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,180] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,183] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,185] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,186] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,189] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,190] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,190] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:12,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:12,192] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:12,192] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,193] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:12,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,194] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,194] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:12,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:12,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:12,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:12,195] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:12,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:12,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:12,196] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 5: [2023-05-13 12:05:12,197] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,197] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:12,198] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,198] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,199] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,207] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,211] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,213] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,213] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,221] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,225] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,226] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,231] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,233] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,233] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,235] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,237] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,241] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,245] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,247] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,248] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,249] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,252] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,253] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,254] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,256] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,256] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,257] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,258] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,258] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:12,259] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:12,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,259] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:12,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,261] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,261] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:12,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 10: [2023-05-13 12:05:12,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,262] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:12,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:12,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:12,263] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,264] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:12,265] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:12,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,266] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,268] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,269] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,270] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,275] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,278] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,280] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 3: [2023-05-13 12:05:12,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,281] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,282] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:12,283] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,285] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,286] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,287] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,288] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,289] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,290] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,291] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,294] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,294] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,295] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 10: [2023-05-13 12:05:12,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,296] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,297] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,298] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,300] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,302] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,303] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,304] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,305] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:12,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,314] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,315] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,320] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,321] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:12,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:12,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:12,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:12,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:12,321] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:12,322] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,334] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:12,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,334] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,335] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,337] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:12,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,338] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:12,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,340] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,341] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,342] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:12,343] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,343] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,344] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:12,346] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,346] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,347] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:12,348] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,348] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:12,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:12,349] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,350] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:12,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:12,351] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,352] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,353] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,355] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,356] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:12,357] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,358] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:12,360] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:12,361] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:12,361] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:12,361] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:12,361] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,360] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,361] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,362] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,362] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,362] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,363] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,364] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,365] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,367] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 14: [2023-05-13 12:05:12,369] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,370] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,371] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,372] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,374] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 14: [2023-05-13 12:05:12,375] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,376] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,377] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,380] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,382] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,382] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 0: [2023-05-13 12:05:12,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,383] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,384] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,384] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,384] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,384] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 26: [2023-05-13 12:05:12,384] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:12,384] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:12,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:12,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:12,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:12,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 0: [2023-05-13 12:05:12,385] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:12,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 23: [2023-05-13 12:05:12,386] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:12,387] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 8: [2023-05-13 12:05:12,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,388] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,389] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,390] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,392] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 2: [2023-05-13 12:05:12,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,395] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,396] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,398] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,399] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 2: [2023-05-13 12:05:12,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,400] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:12,402] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:12,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:12,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:12,403] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,403] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:12,404] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:12,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:12,405] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,406] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,406] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,407] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:12,407] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,408] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 6: [2023-05-13 12:05:12,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,409] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:12,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:12,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 21: [2023-05-13 12:05:12,410] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 12: [2023-05-13 12:05:12,410] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,411] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 11: [2023-05-13 12:05:12,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 6: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 7: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 7: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,413] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:12,415] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,416] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 26: [2023-05-13 12:05:12,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:12,417] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:12,418] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,418] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,419] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:12,419] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:12,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:12,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:12,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:12,421] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 25: [2023-05-13 12:05:12,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,421] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:12,422] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:12,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,423] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,424] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 13: [2023-05-13 12:05:12,425] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,425] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 2: [2023-05-13 12:05:12,427] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,428] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,432] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,433] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 2: [2023-05-13 12:05:12,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 2: [2023-05-13 12:05:12,434] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,434] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:12,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:12,436] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:12,437] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,438] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,438] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,439] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,439] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,440] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,441] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,442] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,442] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 11: [2023-05-13 12:05:12,443] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,443] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,445] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,445] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 12: [2023-05-13 12:05:12,446] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,447] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,448] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,448] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,449] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,450] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,451] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,452] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,453] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,453] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,454] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 25: [2023-05-13 12:05:12,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,454] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,455] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,456] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,456] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,457] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,458] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,459] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 20: [2023-05-13 12:05:12,460] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,461] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:12,461] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,462] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:12,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 24: [2023-05-13 12:05:12,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,463] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,463] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,464] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 29: [2023-05-13 12:05:12,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,465] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 17: [2023-05-13 12:05:12,466] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,469] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 13: [2023-05-13 12:05:12,470] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,471] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,472] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,473] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,474] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,475] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,476] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 21: [2023-05-13 12:05:12,477] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 23: [2023-05-13 12:05:12,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,477] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 29: [2023-05-13 12:05:12,478] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,479] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,481] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,482] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,483] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 4: [2023-05-13 12:05:12,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,484] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,485] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:12,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:12,486] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:12,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,487] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:12,487] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:12,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:12,488] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 18: [2023-05-13 12:05:12,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 8: [2023-05-13 12:05:12,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,489] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,489] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,490] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,491] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,492] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,493] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,495] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 27: [2023-05-13 12:05:12,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,496] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,496] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,497] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,498] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 17: [2023-05-13 12:05:12,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,499] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,499] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:12,500] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 20: [2023-05-13 12:05:12,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:12,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 19: [2023-05-13 12:05:12,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 19: [2023-05-13 12:05:12,501] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 4: [2023-05-13 12:05:12,501] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 27: [2023-05-13 12:05:12,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:12,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 22: [2023-05-13 12:05:12,502] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:12,502] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 27: [2023-05-13 12:05:12,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 28: [2023-05-13 12:05:12,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,503] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,504] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 18: [2023-05-13 12:05:12,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 9: [2023-05-13 12:05:12,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,507] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,508] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,509] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,510] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,511] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,512] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,513] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,514] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,515] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,515] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,516] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,517] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,521] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,522] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,523] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,523] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,524] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,524] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,527] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 22: [2023-05-13 12:05:12,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,527] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,530] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,533] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,533] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,533] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,534] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,535] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,536] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,537] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,538] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,538] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,539] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 30: [2023-05-13 12:05:12,540] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,540] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 1: [2023-05-13 12:05:12,541] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 9: [2023-05-13 12:05:12,541] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 15: [2023-05-13 12:05:12,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,542] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt... 31: [2023-05-13 12:05:12,543] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,544] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,545] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,546] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,547] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,547] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,548] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,549] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 31: [2023-05-13 12:05:12,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,550] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,552] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 27: [2023-05-13 12:05:12,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 24: [2023-05-13 12:05:12,553] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 31: [2023-05-13 12:05:12,554] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,555] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,555] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,555] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,559] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 24: [2023-05-13 12:05:12,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,560] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,561] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 16: [2023-05-13 12:05:12,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 9: [2023-05-13 12:05:12,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,563] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,564] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,564] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,565] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,566] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,567] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 31: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 27: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 30: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,568] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,571] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,572] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,573] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,573] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,575] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,575] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,576] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,576] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,576] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,576] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,576] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,576] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,576] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:12,577] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,578] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,580] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,581] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,582] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,582] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,583] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,583] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,584] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,584] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,584] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 9: [2023-05-13 12:05:12,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,586] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,587] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,588] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,591] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,591] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:12,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 27: [2023-05-13 12:05:12,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 27: [2023-05-13 12:05:12,592] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,593] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,593] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:12,594] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,595] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:12,596] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 20: [2023-05-13 12:05:12,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,596] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 16: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 16: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 20: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,597] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:12,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 20: [2023-05-13 12:05:12,598] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:12,598] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,600] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,601] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,602] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,603] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,603] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,604] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,605] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,606] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,608] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,611] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,611] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,613] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,614] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,614] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,615] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,615] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,617] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,618] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,619] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,620] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,621] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:12,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 5: [2023-05-13 12:05:12,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,622] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,622] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,623] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 31: [2023-05-13 12:05:12,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_249_mp_rank_00_optim_states.pt... 31: [2023-05-13 12:05:12,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_248_mp_rank_00_optim_states.pt... 31: [2023-05-13 12:05:12,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_250_mp_rank_00_optim_states.pt... 31: [2023-05-13 12:05:12,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_251_mp_rank_00_optim_states.pt... 31: [2023-05-13 12:05:12,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_255_mp_rank_00_optim_states.pt... 31: [2023-05-13 12:05:12,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_254_mp_rank_00_optim_states.pt... 31: [2023-05-13 12:05:12,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_252_mp_rank_00_optim_states.pt... 31: [2023-05-13 12:05:12,626] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_253_mp_rank_00_optim_states.pt... 0: [2023-05-13 12:05:12,627] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,628] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,629] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,630] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,631] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:12,632] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 28: [2023-05-13 12:05:12,632] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,633] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,633] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_74_mp_rank_00_optim_states.pt... 9: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_75_mp_rank_00_optim_states.pt... 9: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_76_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_164_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_161_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_160_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_165_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_166_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_163_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_167_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_162_mp_rank_00_optim_states.pt... 15: [2023-05-13 12:05:12,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,636] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,637] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 28: [2023-05-13 12:05:12,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,638] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 1: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 3: [2023-05-13 12:05:12,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_35-model_00-model_states.pt. 15: [2023-05-13 12:05:12,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,640] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,640] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,641] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,646] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,646] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 28: [2023-05-13 12:05:12,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,647] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,647] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 28: [2023-05-13 12:05:12,648] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,650] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,650] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,654] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,657] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,658] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,659] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 16: [2023-05-13 12:05:12,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_129_mp_rank_00_optim_states.pt... 16: [2023-05-13 12:05:12,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_130_mp_rank_00_optim_states.pt... 16: [2023-05-13 12:05:12,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_131_mp_rank_00_optim_states.pt... 16: [2023-05-13 12:05:12,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_133_mp_rank_00_optim_states.pt... 16: [2023-05-13 12:05:12,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_135_mp_rank_00_optim_states.pt... 16: [2023-05-13 12:05:12,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_128_mp_rank_00_optim_states.pt... 16: [2023-05-13 12:05:12,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_134_mp_rank_00_optim_states.pt... 16: [2023-05-13 12:05:12,660] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_132_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,664] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_227_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,664] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_228_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 28: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_225_mp_rank_00_optim_states.pt... 28: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_231_mp_rank_00_optim_states.pt... 28: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_224_mp_rank_00_optim_states.pt... 28: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_230_mp_rank_00_optim_states.pt... 28: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_229_mp_rank_00_optim_states.pt... 28: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_226_mp_rank_00_optim_states.pt... 10: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,666] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,669] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,669] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 5: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_46_mp_rank_00_optim_states.pt... 5: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_42_mp_rank_00_optim_states.pt... 5: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_40_mp_rank_00_optim_states.pt... 5: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_41_mp_rank_00_optim_states.pt... 15: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 5: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_47_mp_rank_00_optim_states.pt... 5: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_44_mp_rank_00_optim_states.pt... 5: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_43_mp_rank_00_optim_states.pt... 5: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_45_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,670] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 15: [2023-05-13 12:05:12,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 15: [2023-05-13 12:05:12,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 15: [2023-05-13 12:05:12,671] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,671] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,672] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,672] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,673] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,675] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:12,676] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,676] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,677] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 9: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_73_mp_rank_00_optim_states.pt... 9: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_77_mp_rank_00_optim_states.pt... 9: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_78_mp_rank_00_optim_states.pt... 9: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_79_mp_rank_00_optim_states.pt... 9: [2023-05-13 12:05:12,634] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_72_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:12,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,678] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,682] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,682] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,683] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,684] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,685] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,687] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,687] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,690] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,694] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_126_mp_rank_00_optim_states.pt... 15: [2023-05-13 12:05:12,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_122_mp_rank_00_optim_states.pt... 15: [2023-05-13 12:05:12,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_120_mp_rank_00_optim_states.pt... 15: [2023-05-13 12:05:12,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_125_mp_rank_00_optim_states.pt... 15: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_127_mp_rank_00_optim_states.pt... 15: [2023-05-13 12:05:12,695] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_121_mp_rank_00_optim_states.pt... 7: [2023-05-13 12:05:12,695] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 15: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_123_mp_rank_00_optim_states.pt... 15: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_124_mp_rank_00_optim_states.pt... 7: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,696] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,697] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,697] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,698] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,698] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,699] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,699] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,700] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,700] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,701] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,703] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,703] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,704] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,704] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 10: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,705] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,706] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,706] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,707] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,708] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,709] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,710] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,711] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 3: [2023-05-13 12:05:12,712] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,713] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,713] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,714] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 10: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 10: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,715] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,716] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,718] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,719] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,719] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,721] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,722] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,723] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,724] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,725] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,725] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,726] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,726] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,728] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,729] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,729] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,730] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,733] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,734] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,734] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,735] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,735] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,736] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 3: [2023-05-13 12:05:12,736] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,737] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,737] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,738] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,738] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 3: [2023-05-13 12:05:12,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,739] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: [2023-05-13 12:05:12,739] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,740] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,741] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,741] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 6: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,742] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 6: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 6: [2023-05-13 12:05:12,743] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,744] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,747] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,748] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 0: [2023-05-13 12:05:12,750] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,749] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,750] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,751] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,752] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,752] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 26: [2023-05-13 12:05:12,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,754] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,754] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,755] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,756] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 0: > overriding learning rate value to 0.0002 0: > overriding minimum learning rate value to 2e-05 0: > overriding warmup iterations value to 0 0: > overriding total number of iterations value to 1 0: > overriding decay style value to cosine 22: [2023-05-13 12:05:12,757] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,757] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,761] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,762] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,762] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 14: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,767] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,768] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 23: [2023-05-13 12:05:12,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,769] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,769] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,770] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,770] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,771] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,772] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 10: [2023-05-13 12:05:12,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_80_mp_rank_00_optim_states.pt... 10: [2023-05-13 12:05:12,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_81_mp_rank_00_optim_states.pt... 10: [2023-05-13 12:05:12,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_87_mp_rank_00_optim_states.pt... 10: [2023-05-13 12:05:12,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_86_mp_rank_00_optim_states.pt... 10: [2023-05-13 12:05:12,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_85_mp_rank_00_optim_states.pt... 10: [2023-05-13 12:05:12,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_83_mp_rank_00_optim_states.pt... 10: [2023-05-13 12:05:12,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_82_mp_rank_00_optim_states.pt... 10: [2023-05-13 12:05:12,773] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_84_mp_rank_00_optim_states.pt... 8: [2023-05-13 12:05:12,774] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,775] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,775] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,776] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,777] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,778] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,779] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,779] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,781] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,782] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,784] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,785] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,786] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,786] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 0: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt... 0: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt... 0: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt... 0: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt... 0: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt... 0: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt... 0: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt... 0: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt... 21: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 6: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_54_mp_rank_00_optim_states.pt... 6: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_51_mp_rank_00_optim_states.pt... 6: [2023-05-13 12:05:12,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_49_mp_rank_00_optim_states.pt... 6: [2023-05-13 12:05:12,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_53_mp_rank_00_optim_states.pt... 6: [2023-05-13 12:05:12,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_48_mp_rank_00_optim_states.pt... 6: [2023-05-13 12:05:12,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_52_mp_rank_00_optim_states.pt... 6: [2023-05-13 12:05:12,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_50_mp_rank_00_optim_states.pt... 6: [2023-05-13 12:05:12,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_55_mp_rank_00_optim_states.pt... 14: [2023-05-13 12:05:12,788] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,787] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,788] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,788] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,789] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,789] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,790] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,790] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,791] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,791] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,792] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,792] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,793] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 14: [2023-05-13 12:05:12,793] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,794] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,795] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,796] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,796] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,797] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,799] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,799] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 3: [2023-05-13 12:05:12,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_26_mp_rank_00_optim_states.pt... 3: [2023-05-13 12:05:12,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_31_mp_rank_00_optim_states.pt... 3: [2023-05-13 12:05:12,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_27_mp_rank_00_optim_states.pt... 3: [2023-05-13 12:05:12,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_28_mp_rank_00_optim_states.pt... 3: [2023-05-13 12:05:12,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_24_mp_rank_00_optim_states.pt... 3: [2023-05-13 12:05:12,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_25_mp_rank_00_optim_states.pt... 3: [2023-05-13 12:05:12,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_29_mp_rank_00_optim_states.pt... 3: [2023-05-13 12:05:12,801] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_30_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,802] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,802] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,802] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,804] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,805] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,805] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,763] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,764] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,765] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 26: [2023-05-13 12:05:12,765] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,766] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,806] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,806] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,807] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,808] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,808] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 23: [2023-05-13 12:05:12,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 23: [2023-05-13 12:05:12,809] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,809] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,810] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,811] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,814] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,814] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 2: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,815] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,817] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,816] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,817] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,818] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,818] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,819] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,820] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,820] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,820] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,820] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,821] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,822] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,822] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,822] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,823] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 22: [2023-05-13 12:05:12,823] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,824] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 12: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 12: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 12: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 2: [2023-05-13 12:05:12,825] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 2: [2023-05-13 12:05:12,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,826] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,826] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 2: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 29: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,827] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,828] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,829] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 7: [2023-05-13 12:05:12,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,830] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,830] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 17: [2023-05-13 12:05:12,831] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,832] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,832] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,833] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,833] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,833] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,834] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,834] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,835] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,835] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,836] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 11: [2023-05-13 12:05:12,837] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,837] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,838] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 7: [2023-05-13 12:05:12,838] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 7: [2023-05-13 12:05:12,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,839] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,839] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,840] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 14: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_116_mp_rank_00_optim_states.pt... 14: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_112_mp_rank_00_optim_states.pt... 14: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_115_mp_rank_00_optim_states.pt... 14: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_113_mp_rank_00_optim_states.pt... 8: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 14: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_118_mp_rank_00_optim_states.pt... 14: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_117_mp_rank_00_optim_states.pt... 14: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_114_mp_rank_00_optim_states.pt... 14: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_119_mp_rank_00_optim_states.pt... 11: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,841] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,842] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,842] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,842] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,843] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,843] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,844] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,844] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,844] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,844] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,845] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 11: [2023-05-13 12:05:12,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,846] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,847] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 11: [2023-05-13 12:05:12,847] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 26: [2023-05-13 12:05:12,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_214_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_215_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_212_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_210_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_211_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_208_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_209_mp_rank_00_optim_states.pt... 26: [2023-05-13 12:05:12,840] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_213_mp_rank_00_optim_states.pt... 22: [2023-05-13 12:05:12,850] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,850] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,850] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,850] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,851] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,852] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,852] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,853] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,853] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,854] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,855] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,855] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,857] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 24: [2023-05-13 12:05:12,857] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 24: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 24: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,858] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 13: [2023-05-13 12:05:12,860] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,861] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,861] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,863] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 8: [2023-05-13 12:05:12,863] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,864] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 24: [2023-05-13 12:05:12,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,864] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 13: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,865] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,866] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,866] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 21: [2023-05-13 12:05:12,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,867] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,868] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,868] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 7: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_63_mp_rank_00_optim_states.pt... 7: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_61_mp_rank_00_optim_states.pt... 7: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_58_mp_rank_00_optim_states.pt... 7: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_60_mp_rank_00_optim_states.pt... 7: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_62_mp_rank_00_optim_states.pt... 7: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_57_mp_rank_00_optim_states.pt... 7: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_59_mp_rank_00_optim_states.pt... 7: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_56_mp_rank_00_optim_states.pt... 19: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 22: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,870] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,871] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,872] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 22: [2023-05-13 12:05:12,873] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,873] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,877] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,879] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,880] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,881] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,881] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,882] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 4: [2023-05-13 12:05:12,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,883] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:12,884] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,884] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 21: [2023-05-13 12:05:12,885] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,886] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,886] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 1: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 19: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 29: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 23: [2023-05-13 12:05:12,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_185_mp_rank_00_optim_states.pt... 23: [2023-05-13 12:05:12,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_188_mp_rank_00_optim_states.pt... 23: [2023-05-13 12:05:12,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_187_mp_rank_00_optim_states.pt... 23: [2023-05-13 12:05:12,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_189_mp_rank_00_optim_states.pt... 23: [2023-05-13 12:05:12,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_191_mp_rank_00_optim_states.pt... 23: [2023-05-13 12:05:12,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_184_mp_rank_00_optim_states.pt... 23: [2023-05-13 12:05:12,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_190_mp_rank_00_optim_states.pt... 23: [2023-05-13 12:05:12,888] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_186_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,890] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,890] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,891] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,892] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,892] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,892] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,892] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,892] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 12: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_98_mp_rank_00_optim_states.pt... 12: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_102_mp_rank_00_optim_states.pt... 12: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_99_mp_rank_00_optim_states.pt... 12: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_100_mp_rank_00_optim_states.pt... 12: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_101_mp_rank_00_optim_states.pt... 12: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_96_mp_rank_00_optim_states.pt... 12: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_97_mp_rank_00_optim_states.pt... 12: [2023-05-13 12:05:12,893] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_103_mp_rank_00_optim_states.pt... 25: [2023-05-13 12:05:12,882] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,887] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,888] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,894] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,895] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,895] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,896] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 18: [2023-05-13 12:05:12,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,896] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,897] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,898] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 30: [2023-05-13 12:05:12,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 4: [2023-05-13 12:05:12,899] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,899] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 13: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_108_mp_rank_00_optim_states.pt... 13: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_111_mp_rank_00_optim_states.pt... 13: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_105_mp_rank_00_optim_states.pt... 13: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_107_mp_rank_00_optim_states.pt... 13: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_104_mp_rank_00_optim_states.pt... 13: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_106_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,901] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,902] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,902] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,903] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,904] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,904] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,905] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,906] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,907] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,910] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,912] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,912] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,913] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,913] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,914] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,914] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,915] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,915] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 4: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 4: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 24: [2023-05-13 12:05:12,916] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 2: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 11: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_90_mp_rank_00_optim_states.pt... 11: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_94_mp_rank_00_optim_states.pt... 11: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_89_mp_rank_00_optim_states.pt... 11: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_92_mp_rank_00_optim_states.pt... 11: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_88_mp_rank_00_optim_states.pt... 11: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_93_mp_rank_00_optim_states.pt... 11: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_95_mp_rank_00_optim_states.pt... 11: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_91_mp_rank_00_optim_states.pt... 2: [2023-05-13 12:05:12,917] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,874] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 8: [2023-05-13 12:05:12,875] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 8: [2023-05-13 12:05:12,875] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,918] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,919] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,919] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,920] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,920] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,921] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 24: [2023-05-13 12:05:12,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 24: [2023-05-13 12:05:12,924] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,925] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,925] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 17: [2023-05-13 12:05:12,926] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,927] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,930] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,930] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,931] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 25: [2023-05-13 12:05:12,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:12,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 27: [2023-05-13 12:05:12,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,932] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,932] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:12,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 27: [2023-05-13 12:05:12,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,933] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:12,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 27: [2023-05-13 12:05:12,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 27: [2023-05-13 12:05:12,933] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:12,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt... 25: [2023-05-13 12:05:12,934] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,934] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 25: [2023-05-13 12:05:12,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,935] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,935] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,936] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,936] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 22: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_177_mp_rank_00_optim_states.pt... 22: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_178_mp_rank_00_optim_states.pt... 22: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_181_mp_rank_00_optim_states.pt... 22: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_183_mp_rank_00_optim_states.pt... 22: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_179_mp_rank_00_optim_states.pt... 22: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_176_mp_rank_00_optim_states.pt... 22: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_180_mp_rank_00_optim_states.pt... 22: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_182_mp_rank_00_optim_states.pt... 17: [2023-05-13 12:05:12,938] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,939] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 19: [2023-05-13 12:05:12,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,940] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 19: [2023-05-13 12:05:12,940] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 13: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_110_mp_rank_00_optim_states.pt... 13: [2023-05-13 12:05:12,900] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_109_mp_rank_00_optim_states.pt... 17: [2023-05-13 12:05:12,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,941] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,941] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 18: [2023-05-13 12:05:12,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,942] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,942] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,943] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,943] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,944] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,944] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 18: [2023-05-13 12:05:12,945] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 18: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_169_mp_rank_00_optim_states.pt... 21: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_173_mp_rank_00_optim_states.pt... 21: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_168_mp_rank_00_optim_states.pt... 21: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_171_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 21: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_174_mp_rank_00_optim_states.pt... 21: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_170_mp_rank_00_optim_states.pt... 21: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_175_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:12,947] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 21: [2023-05-13 12:05:12,946] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_172_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:12,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,947] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,948] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 24: [2023-05-13 12:05:12,952] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,952] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 24: [2023-05-13 12:05:12,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,952] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,953] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,955] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,955] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,957] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,958] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,958] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,959] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,959] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,959] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 8: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_70_mp_rank_00_optim_states.pt... 8: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_66_mp_rank_00_optim_states.pt... 8: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_65_mp_rank_00_optim_states.pt... 8: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_71_mp_rank_00_optim_states.pt... 8: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_67_mp_rank_00_optim_states.pt... 8: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_69_mp_rank_00_optim_states.pt... 8: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_64_mp_rank_00_optim_states.pt... 8: [2023-05-13 12:05:12,937] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_68_mp_rank_00_optim_states.pt... 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:12,962] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 17: [2023-05-13 12:05:12,963] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 29: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,922] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 29: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 2: [2023-05-13 12:05:12,923] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 19: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_156_mp_rank_00_optim_states.pt... 19: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_158_mp_rank_00_optim_states.pt... 19: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_153_mp_rank_00_optim_states.pt... 19: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_154_mp_rank_00_optim_states.pt... 19: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_152_mp_rank_00_optim_states.pt... 19: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_157_mp_rank_00_optim_states.pt... 19: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_159_mp_rank_00_optim_states.pt... 19: [2023-05-13 12:05:12,972] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_155_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_36_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_34_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_39_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_35_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_38_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_32_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_33_mp_rank_00_optim_states.pt... 4: [2023-05-13 12:05:12,973] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_37_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:12,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,975] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:12,975] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:12,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 1: [2023-05-13 12:05:12,977] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,980] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,981] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,981] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,983] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,984] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,984] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,985] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,986] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,986] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,986] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,987] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,987] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,988] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:12,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,990] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,991] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_253_mp_rank_00_optim_states.pt. 30: [2023-05-13 12:05:12,991] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 31: [2023-05-13 12:05:12,992] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 253 30: [2023-05-13 12:05:12,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,992] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,992] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,994] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,994] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,995] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:12,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 27: [2023-05-13 12:05:12,995] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_36-model_00-model_states.pt. 30: [2023-05-13 12:05:12,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 30: [2023-05-13 12:05:12,996] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 30: [2023-05-13 12:05:12,996] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,956] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,959] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,960] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,966] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 24: [2023-05-13 12:05:12,966] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 24: [2023-05-13 12:05:12,967] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 25: [2023-05-13 12:05:12,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_200_mp_rank_00_optim_states.pt... 25: [2023-05-13 12:05:12,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_205_mp_rank_00_optim_states.pt... 25: [2023-05-13 12:05:12,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_204_mp_rank_00_optim_states.pt... 25: [2023-05-13 12:05:12,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_206_mp_rank_00_optim_states.pt... 25: [2023-05-13 12:05:12,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_207_mp_rank_00_optim_states.pt... 25: [2023-05-13 12:05:12,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_203_mp_rank_00_optim_states.pt... 25: [2023-05-13 12:05:12,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_202_mp_rank_00_optim_states.pt... 25: [2023-05-13 12:05:12,998] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_201_mp_rank_00_optim_states.pt... 31: [2023-05-13 12:05:13,000] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 253 18: [2023-05-13 12:05:13,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_151_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:13,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_146_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:13,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_147_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:13,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_149_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:13,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_144_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:13,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_150_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:13,001] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_145_mp_rank_00_optim_states.pt... 18: [2023-05-13 12:05:13,002] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_148_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:13,002] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_161_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,002] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 161 1: [2023-05-13 12:05:13,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,007] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,007] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,008] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 20: [2023-05-13 12:05:13,008] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 161 29: [2023-05-13 12:05:13,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_232_mp_rank_00_optim_states.pt... 29: [2023-05-13 12:05:13,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_236_mp_rank_00_optim_states.pt... 29: [2023-05-13 12:05:13,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_237_mp_rank_00_optim_states.pt... 29: [2023-05-13 12:05:13,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_235_mp_rank_00_optim_states.pt... 29: [2023-05-13 12:05:13,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_234_mp_rank_00_optim_states.pt... 29: [2023-05-13 12:05:13,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_233_mp_rank_00_optim_states.pt... 29: [2023-05-13 12:05:13,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_238_mp_rank_00_optim_states.pt... 29: [2023-05-13 12:05:13,006] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_239_mp_rank_00_optim_states.pt... 2: [2023-05-13 12:05:13,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_20_mp_rank_00_optim_states.pt... 2: [2023-05-13 12:05:13,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_17_mp_rank_00_optim_states.pt... 2: [2023-05-13 12:05:13,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_22_mp_rank_00_optim_states.pt... 2: [2023-05-13 12:05:13,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_23_mp_rank_00_optim_states.pt... 2: [2023-05-13 12:05:13,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_18_mp_rank_00_optim_states.pt... 2: [2023-05-13 12:05:13,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_21_mp_rank_00_optim_states.pt... 2: [2023-05-13 12:05:13,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_19_mp_rank_00_optim_states.pt... 2: [2023-05-13 12:05:13,000] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_16_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:13,011] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,012] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,012] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,015] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,016] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,016] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,017] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,019] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,020] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,020] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,021] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,022] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 1: [2023-05-13 12:05:13,022] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 1: [2023-05-13 12:05:13,022] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,023] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 17: [2023-05-13 12:05:13,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_141_mp_rank_00_optim_states.pt... 17: [2023-05-13 12:05:13,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_137_mp_rank_00_optim_states.pt... 17: [2023-05-13 12:05:13,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_142_mp_rank_00_optim_states.pt... 17: [2023-05-13 12:05:13,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_138_mp_rank_00_optim_states.pt... 9: [2023-05-13 12:05:13,025] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_79_mp_rank_00_optim_states.pt. 17: [2023-05-13 12:05:13,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_139_mp_rank_00_optim_states.pt... 17: [2023-05-13 12:05:13,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_143_mp_rank_00_optim_states.pt... 17: [2023-05-13 12:05:13,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_140_mp_rank_00_optim_states.pt... 17: [2023-05-13 12:05:13,025] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_136_mp_rank_00_optim_states.pt... 9: [2023-05-13 12:05:13,025] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 79 27: [2023-05-13 12:05:13,026] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,027] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 16: [2023-05-13 12:05:13,028] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_134_mp_rank_00_optim_states.pt. 16: [2023-05-13 12:05:13,029] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 134 27: [2023-05-13 12:05:13,029] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,029] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,031] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,031] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt... 27: [2023-05-13 12:05:13,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 27: [2023-05-13 12:05:13,032] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/layer_38-model_00-model_states.pt. 9: [2023-05-13 12:05:13,031] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 79 16: [2023-05-13 12:05:13,033] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 134 24: [2023-05-13 12:05:13,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_193_mp_rank_00_optim_states.pt... 24: [2023-05-13 12:05:13,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_197_mp_rank_00_optim_states.pt... 24: [2023-05-13 12:05:13,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_194_mp_rank_00_optim_states.pt... 24: [2023-05-13 12:05:13,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_192_mp_rank_00_optim_states.pt... 24: [2023-05-13 12:05:13,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_199_mp_rank_00_optim_states.pt... 24: [2023-05-13 12:05:13,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_195_mp_rank_00_optim_states.pt... 24: [2023-05-13 12:05:13,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_196_mp_rank_00_optim_states.pt... 24: [2023-05-13 12:05:13,036] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_198_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:13,042] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_164_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,042] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 164 20: [2023-05-13 12:05:13,047] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 164 31: [2023-05-13 12:05:13,065] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_248_mp_rank_00_optim_states.pt. 31: [2023-05-13 12:05:13,065] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 248 31: [2023-05-13 12:05:13,071] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 248 30: [2023-05-13 12:05:13,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_246_mp_rank_00_optim_states.pt... 30: [2023-05-13 12:05:13,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_244_mp_rank_00_optim_states.pt... 30: [2023-05-13 12:05:13,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_243_mp_rank_00_optim_states.pt... 30: [2023-05-13 12:05:13,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_245_mp_rank_00_optim_states.pt... 30: [2023-05-13 12:05:13,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_242_mp_rank_00_optim_states.pt... 30: [2023-05-13 12:05:13,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_240_mp_rank_00_optim_states.pt... 30: [2023-05-13 12:05:13,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_241_mp_rank_00_optim_states.pt... 30: [2023-05-13 12:05:13,072] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_247_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:13,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_13_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:13,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_9_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:13,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_14_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:13,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_11_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:13,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_10_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:13,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_12_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:13,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_15_mp_rank_00_optim_states.pt... 1: [2023-05-13 12:05:13,075] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_8_mp_rank_00_optim_states.pt... 16: [2023-05-13 12:05:13,091] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_128_mp_rank_00_optim_states.pt. 16: [2023-05-13 12:05:13,092] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 128 16: [2023-05-13 12:05:13,097] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 128 20: [2023-05-13 12:05:13,108] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_160_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,109] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 160 27: [2023-05-13 12:05:13,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_216_mp_rank_00_optim_states.pt... 27: [2023-05-13 12:05:13,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_219_mp_rank_00_optim_states.pt... 27: [2023-05-13 12:05:13,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_222_mp_rank_00_optim_states.pt... 27: [2023-05-13 12:05:13,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_221_mp_rank_00_optim_states.pt... 20: [2023-05-13 12:05:13,115] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 160 20: [2023-05-13 12:05:13,117] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_166_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,117] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 166 15: [2023-05-13 12:05:13,122] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_125_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,123] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 166 15: [2023-05-13 12:05:13,123] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 125 31: [2023-05-13 12:05:13,126] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_249_mp_rank_00_optim_states.pt. 31: [2023-05-13 12:05:13,127] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 249 15: [2023-05-13 12:05:13,129] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 125 31: [2023-05-13 12:05:13,131] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 249 5: [2023-05-13 12:05:13,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_44_mp_rank_00_optim_states.pt. 16: [2023-05-13 12:05:13,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_133_mp_rank_00_optim_states.pt. 5: [2023-05-13 12:05:13,133] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 44 9: [2023-05-13 12:05:13,132] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_74_mp_rank_00_optim_states.pt. 16: [2023-05-13 12:05:13,133] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 133 9: [2023-05-13 12:05:13,133] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 74 28: [2023-05-13 12:05:13,135] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_228_mp_rank_00_optim_states.pt. 28: [2023-05-13 12:05:13,135] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 228 16: [2023-05-13 12:05:13,137] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 133 5: [2023-05-13 12:05:13,138] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 44 9: [2023-05-13 12:05:13,138] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 74 28: [2023-05-13 12:05:13,141] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 228 9: [2023-05-13 12:05:13,143] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_75_mp_rank_00_optim_states.pt. 9: [2023-05-13 12:05:13,143] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 75 31: [2023-05-13 12:05:13,145] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_251_mp_rank_00_optim_states.pt. 31: [2023-05-13 12:05:13,145] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 251 5: [2023-05-13 12:05:13,146] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_43_mp_rank_00_optim_states.pt. 5: [2023-05-13 12:05:13,147] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 43 9: [2023-05-13 12:05:13,149] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 75 31: [2023-05-13 12:05:13,152] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 251 5: [2023-05-13 12:05:13,152] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 43 15: [2023-05-13 12:05:13,153] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_123_mp_rank_00_optim_states.pt. 15: [2023-05-13 12:05:13,153] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 123 27: [2023-05-13 12:05:13,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_223_mp_rank_00_optim_states.pt... 27: [2023-05-13 12:05:13,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_218_mp_rank_00_optim_states.pt... 27: [2023-05-13 12:05:13,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_217_mp_rank_00_optim_states.pt... 27: [2023-05-13 12:05:13,110] [INFO] [torch_checkpoint_engine.py:21:load] [Torch] Loading checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_220_mp_rank_00_optim_states.pt... 28: [2023-05-13 12:05:13,154] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_230_mp_rank_00_optim_states.pt. 28: [2023-05-13 12:05:13,154] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 230 15: [2023-05-13 12:05:13,159] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 123 5: [2023-05-13 12:05:13,159] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_45_mp_rank_00_optim_states.pt. 28: [2023-05-13 12:05:13,159] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 230 16: [2023-05-13 12:05:13,160] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_135_mp_rank_00_optim_states.pt. 5: [2023-05-13 12:05:13,160] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 45 16: [2023-05-13 12:05:13,160] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 135 20: [2023-05-13 12:05:13,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_165_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,161] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 165 20: [2023-05-13 12:05:13,161] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_162_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,162] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 162 5: [2023-05-13 12:05:13,164] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 45 16: [2023-05-13 12:05:13,166] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 135 20: [2023-05-13 12:05:13,168] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 162 20: [2023-05-13 12:05:13,168] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 165 31: [2023-05-13 12:05:13,169] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_254_mp_rank_00_optim_states.pt. 31: [2023-05-13 12:05:13,169] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 254 16: [2023-05-13 12:05:13,173] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_129_mp_rank_00_optim_states.pt. 16: [2023-05-13 12:05:13,173] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 129 31: [2023-05-13 12:05:13,175] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 254 16: [2023-05-13 12:05:13,178] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 129 9: [2023-05-13 12:05:13,178] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_76_mp_rank_00_optim_states.pt. 9: [2023-05-13 12:05:13,179] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 76 28: [2023-05-13 12:05:13,181] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_224_mp_rank_00_optim_states.pt. 28: [2023-05-13 12:05:13,182] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 224 9: [2023-05-13 12:05:13,185] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 76 15: [2023-05-13 12:05:13,187] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_127_mp_rank_00_optim_states.pt. 15: [2023-05-13 12:05:13,187] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 127 28: [2023-05-13 12:05:13,188] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 224 0: [2023-05-13 12:05:13,191] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt. 0: [2023-05-13 12:05:13,192] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 5 15: [2023-05-13 12:05:13,193] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 127 0: [2023-05-13 12:05:13,197] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 5 20: [2023-05-13 12:05:13,202] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_163_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,202] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 163 6: [2023-05-13 12:05:13,206] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_50_mp_rank_00_optim_states.pt. 6: [2023-05-13 12:05:13,207] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 50 10: [2023-05-13 12:05:13,207] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_82_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,208] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 163 10: [2023-05-13 12:05:13,208] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 82 28: [2023-05-13 12:05:13,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_231_mp_rank_00_optim_states.pt. 28: [2023-05-13 12:05:13,210] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 231 26: [2023-05-13 12:05:13,210] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_214_mp_rank_00_optim_states.pt. 26: [2023-05-13 12:05:13,211] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 214 16: [2023-05-13 12:05:13,212] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_132_mp_rank_00_optim_states.pt. 16: [2023-05-13 12:05:13,213] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 132 10: [2023-05-13 12:05:13,213] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 82 6: [2023-05-13 12:05:13,214] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 50 26: [2023-05-13 12:05:13,216] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 214 28: [2023-05-13 12:05:13,216] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 231 9: [2023-05-13 12:05:13,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_73_mp_rank_00_optim_states.pt. 9: [2023-05-13 12:05:13,218] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 73 28: [2023-05-13 12:05:13,218] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_227_mp_rank_00_optim_states.pt. 16: [2023-05-13 12:05:13,218] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 132 28: [2023-05-13 12:05:13,219] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 227 15: [2023-05-13 12:05:13,219] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_124_mp_rank_00_optim_states.pt. 15: [2023-05-13 12:05:13,220] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 124 28: [2023-05-13 12:05:13,221] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_229_mp_rank_00_optim_states.pt. 28: [2023-05-13 12:05:13,222] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 229 28: [2023-05-13 12:05:13,223] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 227 9: [2023-05-13 12:05:13,224] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 73 15: [2023-05-13 12:05:13,226] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 124 28: [2023-05-13 12:05:13,227] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 229 16: [2023-05-13 12:05:13,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_131_mp_rank_00_optim_states.pt. 9: [2023-05-13 12:05:13,228] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_78_mp_rank_00_optim_states.pt. 16: [2023-05-13 12:05:13,228] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 131 9: [2023-05-13 12:05:13,229] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 78 5: [2023-05-13 12:05:13,232] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_41_mp_rank_00_optim_states.pt. 5: [2023-05-13 12:05:13,233] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 41 16: [2023-05-13 12:05:13,234] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 131 9: [2023-05-13 12:05:13,234] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 78 5: [2023-05-13 12:05:13,238] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 41 5: [2023-05-13 12:05:13,240] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_47_mp_rank_00_optim_states.pt. 5: [2023-05-13 12:05:13,241] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 47 9: [2023-05-13 12:05:13,241] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_77_mp_rank_00_optim_states.pt. 9: [2023-05-13 12:05:13,241] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 77 28: [2023-05-13 12:05:13,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_225_mp_rank_00_optim_states.pt. 28: [2023-05-13 12:05:13,243] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 225 15: [2023-05-13 12:05:13,243] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_126_mp_rank_00_optim_states.pt. 15: [2023-05-13 12:05:13,244] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 126 5: [2023-05-13 12:05:13,246] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 47 9: [2023-05-13 12:05:13,247] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 77 28: [2023-05-13 12:05:13,248] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 225 15: [2023-05-13 12:05:13,250] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 126 31: [2023-05-13 12:05:13,250] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_250_mp_rank_00_optim_states.pt. 31: [2023-05-13 12:05:13,250] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 250 13: [2023-05-13 12:05:13,254] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_110_mp_rank_00_optim_states.pt. 13: [2023-05-13 12:05:13,254] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 110 31: [2023-05-13 12:05:13,256] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 250 10: [2023-05-13 12:05:13,260] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_84_mp_rank_00_optim_states.pt. 13: [2023-05-13 12:05:13,261] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 110 10: [2023-05-13 12:05:13,261] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 84 31: [2023-05-13 12:05:13,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_252_mp_rank_00_optim_states.pt. 31: [2023-05-13 12:05:13,262] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 252 16: [2023-05-13 12:05:13,262] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_130_mp_rank_00_optim_states.pt. 16: [2023-05-13 12:05:13,262] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 130 10: [2023-05-13 12:05:13,264] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_81_mp_rank_00_optim_states.pt. 10: [2023-05-13 12:05:13,264] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 81 10: [2023-05-13 12:05:13,267] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 84 31: [2023-05-13 12:05:13,267] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 252 16: [2023-05-13 12:05:13,268] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 130 3: [2023-05-13 12:05:13,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_25_mp_rank_00_optim_states.pt. 10: [2023-05-13 12:05:13,270] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 81 3: [2023-05-13 12:05:13,270] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 25 3: [2023-05-13 12:05:13,270] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_26_mp_rank_00_optim_states.pt. 3: [2023-05-13 12:05:13,271] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 26 3: [2023-05-13 12:05:13,275] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 25 3: [2023-05-13 12:05:13,276] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_27_mp_rank_00_optim_states.pt. 3: [2023-05-13 12:05:13,276] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 26 3: [2023-05-13 12:05:13,277] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 27 15: [2023-05-13 12:05:13,277] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_120_mp_rank_00_optim_states.pt. 15: [2023-05-13 12:05:13,277] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 120 3: [2023-05-13 12:05:13,282] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 27 7: [2023-05-13 12:05:13,282] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_56_mp_rank_00_optim_states.pt. 7: [2023-05-13 12:05:13,282] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 56 15: [2023-05-13 12:05:13,283] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 120 3: [2023-05-13 12:05:13,284] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_28_mp_rank_00_optim_states.pt. 3: [2023-05-13 12:05:13,284] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 28 15: [2023-05-13 12:05:13,286] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_121_mp_rank_00_optim_states.pt. 15: [2023-05-13 12:05:13,286] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 121 7: [2023-05-13 12:05:13,288] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 56 20: [2023-05-13 12:05:13,289] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_167_mp_rank_00_optim_states.pt. 20: [2023-05-13 12:05:13,289] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 167 3: [2023-05-13 12:05:13,289] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 28 6: [2023-05-13 12:05:13,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_52_mp_rank_00_optim_states.pt. 28: [2023-05-13 12:05:13,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_226_mp_rank_00_optim_states.pt. 6: [2023-05-13 12:05:13,290] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 52 28: [2023-05-13 12:05:13,290] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 226 15: [2023-05-13 12:05:13,290] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_122_mp_rank_00_optim_states.pt. 15: [2023-05-13 12:05:13,291] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 122 15: [2023-05-13 12:05:13,292] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 121 12: [2023-05-13 12:05:13,293] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_98_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:13,293] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 98 20: [2023-05-13 12:05:13,294] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 167 6: [2023-05-13 12:05:13,295] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 52 15: [2023-05-13 12:05:13,296] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 122 28: [2023-05-13 12:05:13,296] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 226 23: [2023-05-13 12:05:13,296] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_190_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,297] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 190 12: [2023-05-13 12:05:13,298] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 98 5: [2023-05-13 12:05:13,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_40_mp_rank_00_optim_states.pt. 7: [2023-05-13 12:05:13,300] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_59_mp_rank_00_optim_states.pt. 5: [2023-05-13 12:05:13,301] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 40 7: [2023-05-13 12:05:13,301] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 59 3: [2023-05-13 12:05:13,301] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_24_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,302] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 190 3: [2023-05-13 12:05:13,302] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 24 0: [2023-05-13 12:05:13,302] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt. 0: [2023-05-13 12:05:13,303] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 3 8: [2023-05-13 12:05:13,306] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_71_mp_rank_00_optim_states.pt. 8: [2023-05-13 12:05:13,306] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 71 7: [2023-05-13 12:05:13,306] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 59 5: [2023-05-13 12:05:13,307] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 40 3: [2023-05-13 12:05:13,307] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 24 0: [2023-05-13 12:05:13,309] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 3 8: [2023-05-13 12:05:13,311] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 71 3: [2023-05-13 12:05:13,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_29_mp_rank_00_optim_states.pt. 0: [2023-05-13 12:05:13,312] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt. 3: [2023-05-13 12:05:13,312] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 29 0: [2023-05-13 12:05:13,313] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 0 6: [2023-05-13 12:05:13,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_53_mp_rank_00_optim_states.pt. 0: [2023-05-13 12:05:13,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt. 6: [2023-05-13 12:05:13,313] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 53 0: [2023-05-13 12:05:13,314] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 6 31: [2023-05-13 12:05:13,316] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_255_mp_rank_00_optim_states.pt. 31: [2023-05-13 12:05:13,316] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 255 8: [2023-05-13 12:05:13,317] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_66_mp_rank_00_optim_states.pt. 3: [2023-05-13 12:05:13,317] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 29 8: [2023-05-13 12:05:13,318] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 66 0: [2023-05-13 12:05:13,318] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 0 10: [2023-05-13 12:05:13,318] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_80_mp_rank_00_optim_states.pt. 10: [2023-05-13 12:05:13,318] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 80 6: [2023-05-13 12:05:13,319] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 53 0: [2023-05-13 12:05:13,320] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 6 0: could not find arguments in the checkpoint ... 0: checkpoint version 3.0 8: [2023-05-13 12:05:13,323] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 66 0: [2023-05-13 12:05:13,323] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt. 0: [2023-05-13 12:05:13,323] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt. 3: [2023-05-13 12:05:13,324] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_30_mp_rank_00_optim_states.pt. 31: [2023-05-13 12:05:13,324] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 255 0: [2023-05-13 12:05:13,324] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 4 0: [2023-05-13 12:05:13,324] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 1 10: [2023-05-13 12:05:13,324] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 80 3: [2023-05-13 12:05:13,324] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 30 12: [2023-05-13 12:05:13,325] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_97_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:13,325] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 97 22: [2023-05-13 12:05:13,328] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_180_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,328] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 180 0: [2023-05-13 12:05:13,329] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 4 0: [2023-05-13 12:05:13,330] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 1 3: [2023-05-13 12:05:13,330] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 30 12: [2023-05-13 12:05:13,330] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 97 23: [2023-05-13 12:05:13,330] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_189_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,330] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 189 6: [2023-05-13 12:05:13,331] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_54_mp_rank_00_optim_states.pt. 6: [2023-05-13 12:05:13,332] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 54 6: [2023-05-13 12:05:13,333] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_49_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,333] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 180 6: [2023-05-13 12:05:13,334] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 49 23: [2023-05-13 12:05:13,336] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 189 6: [2023-05-13 12:05:13,337] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 54 26: [2023-05-13 12:05:13,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_209_mp_rank_00_optim_states.pt. 26: [2023-05-13 12:05:13,337] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 209 26: [2023-05-13 12:05:13,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_210_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,337] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_114_mp_rank_00_optim_states.pt. 26: [2023-05-13 12:05:13,337] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 210 14: [2023-05-13 12:05:13,338] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 114 11: [2023-05-13 12:05:13,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_89_mp_rank_00_optim_states.pt. 6: [2023-05-13 12:05:13,339] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 49 7: [2023-05-13 12:05:13,339] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_60_mp_rank_00_optim_states.pt. 11: [2023-05-13 12:05:13,339] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 89 7: [2023-05-13 12:05:13,340] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 60 6: [2023-05-13 12:05:13,340] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_48_mp_rank_00_optim_states.pt. 6: [2023-05-13 12:05:13,341] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 48 26: [2023-05-13 12:05:13,343] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 209 26: [2023-05-13 12:05:13,343] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 210 14: [2023-05-13 12:05:13,343] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 114 11: [2023-05-13 12:05:13,344] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 89 23: [2023-05-13 12:05:13,345] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_185_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,345] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 185 7: [2023-05-13 12:05:13,346] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 60 6: [2023-05-13 12:05:13,346] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 48 11: [2023-05-13 12:05:13,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_90_mp_rank_00_optim_states.pt. 11: [2023-05-13 12:05:13,349] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 90 25: [2023-05-13 12:05:13,349] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_202_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,350] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 202 23: [2023-05-13 12:05:13,350] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 185 14: [2023-05-13 12:05:13,352] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_119_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,352] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 119 11: [2023-05-13 12:05:13,354] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 90 0: [2023-05-13 12:05:13,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,354] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 202 0: [2023-05-13 12:05:13,355] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 2 3: [2023-05-13 12:05:13,356] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_31_mp_rank_00_optim_states.pt. 3: [2023-05-13 12:05:13,357] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 31 6: [2023-05-13 12:05:13,357] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_51_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,358] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 119 6: [2023-05-13 12:05:13,358] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 51 0: [2023-05-13 12:05:13,361] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 2 3: [2023-05-13 12:05:13,362] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 31 6: [2023-05-13 12:05:13,363] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 51 23: [2023-05-13 12:05:13,365] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_186_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,365] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 186 19: [2023-05-13 12:05:13,368] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_155_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,368] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_117_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,368] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 117 19: [2023-05-13 12:05:13,369] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 155 11: [2023-05-13 12:05:13,371] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_88_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,371] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 186 11: [2023-05-13 12:05:13,371] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 88 14: [2023-05-13 12:05:13,374] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 117 19: [2023-05-13 12:05:13,374] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 155 11: [2023-05-13 12:05:13,376] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 88 29: [2023-05-13 12:05:13,378] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_237_mp_rank_00_optim_states.pt. 29: [2023-05-13 12:05:13,378] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 237 8: [2023-05-13 12:05:13,379] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_64_mp_rank_00_optim_states.pt. 8: [2023-05-13 12:05:13,380] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 64 29: [2023-05-13 12:05:13,383] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 237 2: [2023-05-13 12:05:13,386] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_20_mp_rank_00_optim_states.pt. 2: [2023-05-13 12:05:13,386] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 20 8: [2023-05-13 12:05:13,387] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 64 10: [2023-05-13 12:05:13,387] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_86_mp_rank_00_optim_states.pt. 10: [2023-05-13 12:05:13,388] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 86 29: [2023-05-13 12:05:13,388] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_233_mp_rank_00_optim_states.pt. 29: [2023-05-13 12:05:13,389] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 233 14: [2023-05-13 12:05:13,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_113_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,389] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 113 4: [2023-05-13 12:05:13,389] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_33_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,390] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_118_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,390] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 33 14: [2023-05-13 12:05:13,390] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 118 2: [2023-05-13 12:05:13,391] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 20 10: [2023-05-13 12:05:13,393] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 86 29: [2023-05-13 12:05:13,394] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 233 4: [2023-05-13 12:05:13,395] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 33 14: [2023-05-13 12:05:13,395] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 113 14: [2023-05-13 12:05:13,396] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 118 11: [2023-05-13 12:05:13,397] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_95_mp_rank_00_optim_states.pt. 11: [2023-05-13 12:05:13,397] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 95 26: [2023-05-13 12:05:13,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_215_mp_rank_00_optim_states.pt. 26: [2023-05-13 12:05:13,402] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_208_mp_rank_00_optim_states.pt. 26: [2023-05-13 12:05:13,403] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 208 26: [2023-05-13 12:05:13,403] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 215 11: [2023-05-13 12:05:13,404] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 95 23: [2023-05-13 12:05:13,405] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_184_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,405] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 184 26: [2023-05-13 12:05:13,408] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 208 26: [2023-05-13 12:05:13,408] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 215 22: [2023-05-13 12:05:13,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_177_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,410] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 177 23: [2023-05-13 12:05:13,411] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 184 22: [2023-05-13 12:05:13,411] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_182_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,411] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 182 13: [2023-05-13 12:05:13,412] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_105_mp_rank_00_optim_states.pt. 13: [2023-05-13 12:05:13,412] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 105 22: [2023-05-13 12:05:13,415] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 177 22: [2023-05-13 12:05:13,415] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_179_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,416] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 179 22: [2023-05-13 12:05:13,417] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 182 13: [2023-05-13 12:05:13,418] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 105 22: [2023-05-13 12:05:13,421] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 179 29: [2023-05-13 12:05:13,422] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_239_mp_rank_00_optim_states.pt. 29: [2023-05-13 12:05:13,422] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 239 10: [2023-05-13 12:05:13,426] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_87_mp_rank_00_optim_states.pt. 10: [2023-05-13 12:05:13,427] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 87 29: [2023-05-13 12:05:13,427] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 239 2: [2023-05-13 12:05:13,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_16_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,429] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_112_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,429] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 112 2: [2023-05-13 12:05:13,429] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 16 0: [2023-05-13 12:05:13,431] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt. 0: [2023-05-13 12:05:13,432] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 7 10: [2023-05-13 12:05:13,432] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 87 2: [2023-05-13 12:05:13,434] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 16 14: [2023-05-13 12:05:13,435] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 112 21: [2023-05-13 12:05:13,436] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_175_mp_rank_00_optim_states.pt. 21: [2023-05-13 12:05:13,437] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 175 0: [2023-05-13 12:05:13,438] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 7 26: [2023-05-13 12:05:13,440] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_212_mp_rank_00_optim_states.pt. 26: [2023-05-13 12:05:13,440] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 212 13: [2023-05-13 12:05:13,441] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_107_mp_rank_00_optim_states.pt. 13: [2023-05-13 12:05:13,441] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 107 21: [2023-05-13 12:05:13,442] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 175 21: [2023-05-13 12:05:13,443] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_174_mp_rank_00_optim_states.pt. 21: [2023-05-13 12:05:13,444] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 174 26: [2023-05-13 12:05:13,446] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 212 7: [2023-05-13 12:05:13,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_61_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,446] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_178_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,447] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 178 7: [2023-05-13 12:05:13,447] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 61 13: [2023-05-13 12:05:13,449] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 107 21: [2023-05-13 12:05:13,449] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 174 24: [2023-05-13 12:05:13,450] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_194_mp_rank_00_optim_states.pt. 24: [2023-05-13 12:05:13,450] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 194 29: [2023-05-13 12:05:13,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_235_mp_rank_00_optim_states.pt. 21: [2023-05-13 12:05:13,451] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_173_mp_rank_00_optim_states.pt. 29: [2023-05-13 12:05:13,452] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 235 18: [2023-05-13 12:05:13,452] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_149_mp_rank_00_optim_states.pt. 21: [2023-05-13 12:05:13,452] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 173 18: [2023-05-13 12:05:13,452] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 149 22: [2023-05-13 12:05:13,452] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 178 7: [2023-05-13 12:05:13,452] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 61 24: [2023-05-13 12:05:13,455] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 194 21: [2023-05-13 12:05:13,457] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 173 18: [2023-05-13 12:05:13,457] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 149 29: [2023-05-13 12:05:13,457] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 235 8: [2023-05-13 12:05:13,457] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_69_mp_rank_00_optim_states.pt. 8: [2023-05-13 12:05:13,458] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 69 11: [2023-05-13 12:05:13,458] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_94_mp_rank_00_optim_states.pt. 11: [2023-05-13 12:05:13,459] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 94 13: [2023-05-13 12:05:13,459] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_109_mp_rank_00_optim_states.pt. 13: [2023-05-13 12:05:13,460] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 109 19: [2023-05-13 12:05:13,462] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_156_mp_rank_00_optim_states.pt. 19: [2023-05-13 12:05:13,462] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 156 8: [2023-05-13 12:05:13,463] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 69 11: [2023-05-13 12:05:13,464] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 94 12: [2023-05-13 12:05:13,464] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_101_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:13,465] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 101 13: [2023-05-13 12:05:13,466] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 109 8: [2023-05-13 12:05:13,466] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_70_mp_rank_00_optim_states.pt. 19: [2023-05-13 12:05:13,467] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 156 8: [2023-05-13 12:05:13,467] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 70 13: [2023-05-13 12:05:13,467] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_104_mp_rank_00_optim_states.pt. 24: [2023-05-13 12:05:13,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_199_mp_rank_00_optim_states.pt. 13: [2023-05-13 12:05:13,468] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 104 24: [2023-05-13 12:05:13,468] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 199 12: [2023-05-13 12:05:13,468] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_99_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:13,469] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 99 12: [2023-05-13 12:05:13,469] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 101 4: [2023-05-13 12:05:13,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_32_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,469] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_36_mp_rank_00_optim_states.pt. 7: [2023-05-13 12:05:13,470] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_63_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,470] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 32 4: [2023-05-13 12:05:13,470] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 36 7: [2023-05-13 12:05:13,470] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 63 8: [2023-05-13 12:05:13,472] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 70 17: [2023-05-13 12:05:13,472] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_142_mp_rank_00_optim_states.pt. 17: [2023-05-13 12:05:13,472] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 142 24: [2023-05-13 12:05:13,473] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 199 18: [2023-05-13 12:05:13,473] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_151_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:13,473] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 99 18: [2023-05-13 12:05:13,474] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 151 13: [2023-05-13 12:05:13,474] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 104 4: [2023-05-13 12:05:13,475] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 32 7: [2023-05-13 12:05:13,475] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 63 4: [2023-05-13 12:05:13,475] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 36 26: [2023-05-13 12:05:13,476] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_213_mp_rank_00_optim_states.pt. 26: [2023-05-13 12:05:13,476] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 213 17: [2023-05-13 12:05:13,478] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 142 18: [2023-05-13 12:05:13,478] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 151 24: [2023-05-13 12:05:13,478] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_193_mp_rank_00_optim_states.pt. 24: [2023-05-13 12:05:13,479] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 193 4: [2023-05-13 12:05:13,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_38_mp_rank_00_optim_states.pt. 10: [2023-05-13 12:05:13,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_83_mp_rank_00_optim_states.pt. 11: [2023-05-13 12:05:13,480] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_91_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,480] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 38 10: [2023-05-13 12:05:13,481] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 83 11: [2023-05-13 12:05:13,481] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 91 26: [2023-05-13 12:05:13,482] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 213 24: [2023-05-13 12:05:13,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_196_mp_rank_00_optim_states.pt. 7: [2023-05-13 12:05:13,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_62_mp_rank_00_optim_states.pt. 24: [2023-05-13 12:05:13,484] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 196 24: [2023-05-13 12:05:13,484] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 193 7: [2023-05-13 12:05:13,484] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 62 25: [2023-05-13 12:05:13,484] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_201_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,485] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 201 21: [2023-05-13 12:05:13,485] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_169_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,485] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 38 21: [2023-05-13 12:05:13,485] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 169 11: [2023-05-13 12:05:13,486] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 91 2: [2023-05-13 12:05:13,486] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_23_mp_rank_00_optim_states.pt. 10: [2023-05-13 12:05:13,486] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 83 2: [2023-05-13 12:05:13,487] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 23 11: [2023-05-13 12:05:13,488] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_92_mp_rank_00_optim_states.pt. 11: [2023-05-13 12:05:13,489] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 92 24: [2023-05-13 12:05:13,489] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 196 25: [2023-05-13 12:05:13,490] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 201 7: [2023-05-13 12:05:13,490] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 62 8: [2023-05-13 12:05:13,491] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_67_mp_rank_00_optim_states.pt. 21: [2023-05-13 12:05:13,491] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 169 8: [2023-05-13 12:05:13,491] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 67 2: [2023-05-13 12:05:13,492] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 23 11: [2023-05-13 12:05:13,494] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 92 18: [2023-05-13 12:05:13,495] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_146_mp_rank_00_optim_states.pt. 18: [2023-05-13 12:05:13,495] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 146 8: [2023-05-13 12:05:13,496] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 67 2: [2023-05-13 12:05:13,497] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_17_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,498] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_181_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,498] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 181 2: [2023-05-13 12:05:13,498] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 17 18: [2023-05-13 12:05:13,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_150_mp_rank_00_optim_states.pt. 18: [2023-05-13 12:05:13,500] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 150 21: [2023-05-13 12:05:13,500] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_171_mp_rank_00_optim_states.pt. 18: [2023-05-13 12:05:13,500] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 146 21: [2023-05-13 12:05:13,501] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 171 22: [2023-05-13 12:05:13,503] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 181 2: [2023-05-13 12:05:13,503] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 17 23: [2023-05-13 12:05:13,503] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_191_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,504] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 191 14: [2023-05-13 12:05:13,504] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_115_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:13,505] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 115 18: [2023-05-13 12:05:13,505] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 150 21: [2023-05-13 12:05:13,507] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 171 29: [2023-05-13 12:05:13,508] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_232_mp_rank_00_optim_states.pt. 29: [2023-05-13 12:05:13,508] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 232 23: [2023-05-13 12:05:13,508] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 191 12: [2023-05-13 12:05:13,510] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_96_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:13,510] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 96 14: [2023-05-13 12:05:13,510] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 115 29: [2023-05-13 12:05:13,513] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 232 24: [2023-05-13 12:05:13,513] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_192_mp_rank_00_optim_states.pt. 24: [2023-05-13 12:05:13,513] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 192 19: [2023-05-13 12:05:13,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_159_mp_rank_00_optim_states.pt. 17: [2023-05-13 12:05:13,514] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_141_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:13,514] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 96 19: [2023-05-13 12:05:13,515] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 159 17: [2023-05-13 12:05:13,515] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 141 2: [2023-05-13 12:05:13,518] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_21_mp_rank_00_optim_states.pt. 2: [2023-05-13 12:05:13,518] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 21 19: [2023-05-13 12:05:13,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_154_mp_rank_00_optim_states.pt. 19: [2023-05-13 12:05:13,519] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 154 24: [2023-05-13 12:05:13,519] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 192 19: [2023-05-13 12:05:13,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_158_mp_rank_00_optim_states.pt. 2: [2023-05-13 12:05:13,519] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_22_mp_rank_00_optim_states.pt. 19: [2023-05-13 12:05:13,520] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 158 2: [2023-05-13 12:05:13,520] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 22 19: [2023-05-13 12:05:13,520] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 159 17: [2023-05-13 12:05:13,521] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 141 17: [2023-05-13 12:05:13,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_140_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,522] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_39_mp_rank_00_optim_states.pt. 17: [2023-05-13 12:05:13,523] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 140 4: [2023-05-13 12:05:13,523] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 39 2: [2023-05-13 12:05:13,523] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 21 19: [2023-05-13 12:05:13,524] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 154 2: [2023-05-13 12:05:13,526] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 22 19: [2023-05-13 12:05:13,526] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 158 13: [2023-05-13 12:05:13,526] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_108_mp_rank_00_optim_states.pt. 13: [2023-05-13 12:05:13,527] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 108 27: [2023-05-13 12:05:13,528] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_216_mp_rank_00_optim_states.pt. 27: [2023-05-13 12:05:13,528] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 216 4: [2023-05-13 12:05:13,528] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 39 17: [2023-05-13 12:05:13,529] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 140 19: [2023-05-13 12:05:13,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_157_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,531] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_34_mp_rank_00_optim_states.pt. 19: [2023-05-13 12:05:13,532] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 157 13: [2023-05-13 12:05:13,532] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 108 4: [2023-05-13 12:05:13,532] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 34 27: [2023-05-13 12:05:13,533] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 216 11: [2023-05-13 12:05:13,537] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_93_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,537] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 34 11: [2023-05-13 12:05:13,537] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 93 19: [2023-05-13 12:05:13,537] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 157 17: [2023-05-13 12:05:13,542] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_137_mp_rank_00_optim_states.pt. 17: [2023-05-13 12:05:13,542] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 137 11: [2023-05-13 12:05:13,543] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 93 23: [2023-05-13 12:05:13,544] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_187_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,544] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 187 8: [2023-05-13 12:05:13,545] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_68_mp_rank_00_optim_states.pt. 8: [2023-05-13 12:05:13,546] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 68 13: [2023-05-13 12:05:13,546] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_106_mp_rank_00_optim_states.pt. 13: [2023-05-13 12:05:13,547] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 106 17: [2023-05-13 12:05:13,548] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 137 2: [2023-05-13 12:05:13,548] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_18_mp_rank_00_optim_states.pt. 18: [2023-05-13 12:05:13,549] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_147_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:13,549] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 187 2: [2023-05-13 12:05:13,549] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 18 18: [2023-05-13 12:05:13,549] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 147 8: [2023-05-13 12:05:13,551] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 68 13: [2023-05-13 12:05:13,552] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 106 13: [2023-05-13 12:05:13,552] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_111_mp_rank_00_optim_states.pt. 13: [2023-05-13 12:05:13,552] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 111 2: [2023-05-13 12:05:13,554] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 18 18: [2023-05-13 12:05:13,554] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 147 4: [2023-05-13 12:05:13,554] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_35_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,555] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 35 25: [2023-05-13 12:05:13,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_206_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,556] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 206 30: [2023-05-13 12:05:13,556] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_244_mp_rank_00_optim_states.pt. 30: [2023-05-13 12:05:13,556] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 244 13: [2023-05-13 12:05:13,557] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 111 25: [2023-05-13 12:05:13,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_204_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,557] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 204 1: [2023-05-13 12:05:13,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_8_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,557] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_207_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,557] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 207 1: [2023-05-13 12:05:13,558] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 8 4: [2023-05-13 12:05:13,560] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 35 30: [2023-05-13 12:05:13,561] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 244 25: [2023-05-13 12:05:13,561] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 206 25: [2023-05-13 12:05:13,562] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 204 25: [2023-05-13 12:05:13,562] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_205_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,562] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 205 17: [2023-05-13 12:05:13,563] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_143_mp_rank_00_optim_states.pt. 17: [2023-05-13 12:05:13,563] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 143 30: [2023-05-13 12:05:13,563] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_240_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,563] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 207 30: [2023-05-13 12:05:13,563] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 240 1: [2023-05-13 12:05:13,564] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 8 18: [2023-05-13 12:05:13,565] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_148_mp_rank_00_optim_states.pt. 18: [2023-05-13 12:05:13,565] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 148 24: [2023-05-13 12:05:13,566] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_195_mp_rank_00_optim_states.pt. 24: [2023-05-13 12:05:13,567] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 195 25: [2023-05-13 12:05:13,567] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 205 7: [2023-05-13 12:05:13,567] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_57_mp_rank_00_optim_states.pt. 7: [2023-05-13 12:05:13,568] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 57 17: [2023-05-13 12:05:13,569] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 143 30: [2023-05-13 12:05:13,569] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 240 4: [2023-05-13 12:05:13,570] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_37_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,570] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 37 1: [2023-05-13 12:05:13,571] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_9_mp_rank_00_optim_states.pt. 18: [2023-05-13 12:05:13,571] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 148 1: [2023-05-13 12:05:13,571] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 9 30: [2023-05-13 12:05:13,572] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_246_mp_rank_00_optim_states.pt. 30: [2023-05-13 12:05:13,572] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 246 7: [2023-05-13 12:05:13,574] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 57 24: [2023-05-13 12:05:13,574] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 195 1: [2023-05-13 12:05:13,575] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_11_mp_rank_00_optim_states.pt. 4: [2023-05-13 12:05:13,575] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 37 1: [2023-05-13 12:05:13,576] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 11 21: [2023-05-13 12:05:13,576] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_170_mp_rank_00_optim_states.pt. 21: [2023-05-13 12:05:13,576] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 170 1: [2023-05-13 12:05:13,577] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 9 30: [2023-05-13 12:05:13,578] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 246 29: [2023-05-13 12:05:13,580] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_236_mp_rank_00_optim_states.pt. 29: [2023-05-13 12:05:13,580] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 236 21: [2023-05-13 12:05:13,581] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 170 1: [2023-05-13 12:05:13,582] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 11 24: [2023-05-13 12:05:13,585] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_198_mp_rank_00_optim_states.pt. 24: [2023-05-13 12:05:13,585] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 198 29: [2023-05-13 12:05:13,586] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 236 21: [2023-05-13 12:05:13,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_168_mp_rank_00_optim_states.pt. 21: [2023-05-13 12:05:13,590] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 168 29: [2023-05-13 12:05:13,590] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_238_mp_rank_00_optim_states.pt. 29: [2023-05-13 12:05:13,590] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 238 24: [2023-05-13 12:05:13,590] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 198 1: [2023-05-13 12:05:13,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_13_mp_rank_00_optim_states.pt. 27: [2023-05-13 12:05:13,592] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_217_mp_rank_00_optim_states.pt. 27: [2023-05-13 12:05:13,592] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 217 1: [2023-05-13 12:05:13,592] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 13 1: [2023-05-13 12:05:13,594] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_10_mp_rank_00_optim_states.pt. 1: [2023-05-13 12:05:13,595] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 10 21: [2023-05-13 12:05:13,595] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 168 29: [2023-05-13 12:05:13,595] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 238 27: [2023-05-13 12:05:13,597] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 217 1: [2023-05-13 12:05:13,598] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 13 1: [2023-05-13 12:05:13,602] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 10 30: [2023-05-13 12:05:13,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_241_mp_rank_00_optim_states.pt. 30: [2023-05-13 12:05:13,616] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 241 25: [2023-05-13 12:05:13,616] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_200_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,617] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 200 27: [2023-05-13 12:05:13,617] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_218_mp_rank_00_optim_states.pt. 27: [2023-05-13 12:05:13,617] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 218 27: [2023-05-13 12:05:13,618] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_221_mp_rank_00_optim_states.pt. 27: [2023-05-13 12:05:13,618] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 221 27: [2023-05-13 12:05:13,619] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_222_mp_rank_00_optim_states.pt. 27: [2023-05-13 12:05:13,619] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 222 1: [2023-05-13 12:05:13,621] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_14_mp_rank_00_optim_states.pt. 30: [2023-05-13 12:05:13,622] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 241 1: [2023-05-13 12:05:13,622] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 14 25: [2023-05-13 12:05:13,622] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 200 27: [2023-05-13 12:05:13,624] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 218 27: [2023-05-13 12:05:13,627] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 221 27: [2023-05-13 12:05:13,627] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 222 1: [2023-05-13 12:05:13,628] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 14 30: [2023-05-13 12:05:13,628] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_243_mp_rank_00_optim_states.pt. 30: [2023-05-13 12:05:13,628] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 243 27: [2023-05-13 12:05:13,631] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_223_mp_rank_00_optim_states.pt. 27: [2023-05-13 12:05:13,631] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 223 30: [2023-05-13 12:05:13,633] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 243 17: [2023-05-13 12:05:13,635] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_139_mp_rank_00_optim_states.pt. 17: [2023-05-13 12:05:13,635] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 139 27: [2023-05-13 12:05:13,636] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 223 30: [2023-05-13 12:05:13,638] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_242_mp_rank_00_optim_states.pt. 30: [2023-05-13 12:05:13,638] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 242 24: [2023-05-13 12:05:13,639] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_197_mp_rank_00_optim_states.pt. 24: [2023-05-13 12:05:13,639] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 197 17: [2023-05-13 12:05:13,642] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 139 30: [2023-05-13 12:05:13,644] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 242 24: [2023-05-13 12:05:13,645] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 197 27: [2023-05-13 12:05:13,653] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_219_mp_rank_00_optim_states.pt. 27: [2023-05-13 12:05:13,653] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 219 27: [2023-05-13 12:05:13,659] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 219 12: [2023-05-13 12:05:13,661] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_103_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:13,662] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 103 22: [2023-05-13 12:05:13,665] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_183_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:13,665] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 183 12: [2023-05-13 12:05:13,666] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 103 25: [2023-05-13 12:05:13,670] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_203_mp_rank_00_optim_states.pt. 25: [2023-05-13 12:05:13,670] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 203 22: [2023-05-13 12:05:13,671] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 183 8: [2023-05-13 12:05:13,674] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_65_mp_rank_00_optim_states.pt. 8: [2023-05-13 12:05:13,675] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 65 30: [2023-05-13 12:05:13,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_247_mp_rank_00_optim_states.pt. 30: [2023-05-13 12:05:13,677] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 247 18: [2023-05-13 12:05:13,677] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_145_mp_rank_00_optim_states.pt. 18: [2023-05-13 12:05:13,678] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 145 25: [2023-05-13 12:05:13,678] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 203 8: [2023-05-13 12:05:13,680] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 65 29: [2023-05-13 12:05:13,681] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_234_mp_rank_00_optim_states.pt. 29: [2023-05-13 12:05:13,682] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 234 30: [2023-05-13 12:05:13,682] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 247 18: [2023-05-13 12:05:13,684] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 145 29: [2023-05-13 12:05:13,687] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 234 26: [2023-05-13 12:05:13,690] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_211_mp_rank_00_optim_states.pt. 26: [2023-05-13 12:05:13,690] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 211 17: [2023-05-13 12:05:13,694] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_136_mp_rank_00_optim_states.pt. 17: [2023-05-13 12:05:13,694] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 136 26: [2023-05-13 12:05:13,695] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 211 17: [2023-05-13 12:05:13,700] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 136 1: [2023-05-13 12:05:13,702] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_15_mp_rank_00_optim_states.pt. 1: [2023-05-13 12:05:13,703] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 15 21: [2023-05-13 12:05:13,707] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_172_mp_rank_00_optim_states.pt. 21: [2023-05-13 12:05:13,708] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 172 1: [2023-05-13 12:05:13,709] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 15 21: [2023-05-13 12:05:13,712] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 172 30: [2023-05-13 12:05:13,731] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_245_mp_rank_00_optim_states.pt. 30: [2023-05-13 12:05:13,731] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 245 30: [2023-05-13 12:05:13,737] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 245 5: [2023-05-13 12:05:13,751] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_42_mp_rank_00_optim_states.pt. 5: [2023-05-13 12:05:13,751] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 42 5: [2023-05-13 12:05:13,757] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 42 1: [2023-05-13 12:05:13,801] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_12_mp_rank_00_optim_states.pt. 1: [2023-05-13 12:05:13,802] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 12 1: [2023-05-13 12:05:13,808] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 12 7: [2023-05-13 12:05:13,869] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_58_mp_rank_00_optim_states.pt. 7: [2023-05-13 12:05:13,869] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 58 7: [2023-05-13 12:05:13,875] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 58 6: [2023-05-13 12:05:13,960] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_55_mp_rank_00_optim_states.pt. 6: [2023-05-13 12:05:13,960] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 55 6: [2023-05-13 12:05:13,966] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 55 19: [2023-05-13 12:05:14,063] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_153_mp_rank_00_optim_states.pt. 19: [2023-05-13 12:05:14,064] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 153 19: [2023-05-13 12:05:14,070] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 153 12: [2023-05-13 12:05:14,102] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_100_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:14,102] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 100 12: [2023-05-13 12:05:14,108] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 100 10: [2023-05-13 12:05:14,110] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_85_mp_rank_00_optim_states.pt. 10: [2023-05-13 12:05:14,111] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 85 10: [2023-05-13 12:05:14,116] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 85 17: [2023-05-13 12:05:14,184] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_138_mp_rank_00_optim_states.pt. 17: [2023-05-13 12:05:14,184] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 138 17: [2023-05-13 12:05:14,190] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 138 22: [2023-05-13 12:05:14,313] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_176_mp_rank_00_optim_states.pt. 22: [2023-05-13 12:05:14,314] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 176 22: [2023-05-13 12:05:14,318] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 176 27: [2023-05-13 12:05:14,409] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_220_mp_rank_00_optim_states.pt. 27: [2023-05-13 12:05:14,410] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 220 27: [2023-05-13 12:05:14,416] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 220 9: [2023-05-13 12:05:15,819] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_72_mp_rank_00_optim_states.pt. 9: [2023-05-13 12:05:15,820] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 72 9: [2023-05-13 12:05:15,826] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 72 19: [2023-05-13 12:05:16,332] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_152_mp_rank_00_optim_states.pt. 19: [2023-05-13 12:05:16,333] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 152 19: [2023-05-13 12:05:16,340] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 152 14: [2023-05-13 12:05:16,354] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_116_mp_rank_00_optim_states.pt. 14: [2023-05-13 12:05:16,355] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 116 14: [2023-05-13 12:05:16,360] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 116 12: [2023-05-13 12:05:16,482] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_102_mp_rank_00_optim_states.pt. 12: [2023-05-13 12:05:16,482] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 102 12: [2023-05-13 12:05:16,487] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 102 2: [2023-05-13 12:05:16,505] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_19_mp_rank_00_optim_states.pt. 2: [2023-05-13 12:05:16,505] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 19 2: [2023-05-13 12:05:16,510] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 19 5: [2023-05-13 12:05:16,553] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_46_mp_rank_00_optim_states.pt. 5: [2023-05-13 12:05:16,554] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 46 5: [2023-05-13 12:05:16,558] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 46 18: [2023-05-13 12:05:16,595] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_144_mp_rank_00_optim_states.pt. 18: [2023-05-13 12:05:16,595] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 144 18: [2023-05-13 12:05:16,602] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 144 23: [2023-05-13 12:05:17,423] [INFO] [torch_checkpoint_engine.py:23:load] [Torch] Loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50/global_step52452/bf16_zero_pp_rank_188_mp_rank_00_optim_states.pt. 23: [2023-05-13 12:05:17,424] [INFO] [engine.py:2844:_get_all_zero_checkpoint_state_dicts] successfully read 256 ZeRO state_dicts for rank 188 23: [2023-05-13 12:05:17,431] [INFO] [engine.py:2784:_load_zero_checkpoint] loading 256 zero partition checkpoints for rank 188 0: successfully loaded checkpoint from lm1-2b8-55b-c4-perplexity/perplexity50 at iteration 0 31: time (ms) | load-checkpoint: 22270.17 0: estimated model parameters: 2.80902656 0: estimated model parameters without embeddings: 2.67500544 0: [after model, optimizer, and learning rate scheduler are built] datetime: 2023-05-13 12:05:18 0: > building train, validation, and test datasets ... 0: > datasets target sizes (minimum size): 0: train: 1 0: validation: 51200 0: test: 51200 0: > building train, validation, and test datasets for GPT ... 0: > building dataset index ... 0: reading sizes... 0: reading pointers... 0: reading document index... 0: creating numpy buffer of mmap... 0: creating memory view of numpy buffer... 0: > finished creating indexed dataset in 0.062563 seconds 0: number of documents: 3133972 0: > dataset split: 0: train: 0: document indices in [0, 3133972) total of 3133972 documents 0: > loading doc-idx mapping from /scratch/project_462000119/data/c4_subsampled/gpt2tok_c4_en_1B5_text_document_train_indexmap_1ns_2048sl_1234s_doc_idx.npy 0: > loading sample-idx mapping from /scratch/project_462000119/data/c4_subsampled/gpt2tok_c4_en_1B5_text_document_train_indexmap_1ns_2048sl_1234s_sample_idx.npy 0: > loading shuffle-idx mapping from /scratch/project_462000119/data/c4_subsampled/gpt2tok_c4_en_1B5_text_document_train_indexmap_1ns_2048sl_1234s_shuffle_idx.npy 0: loaded indexed file in 0.144 seconds 0: total number of samples: 731002 0: total number of epochs: 1 0: > building dataset index ... 0: reading sizes... 0: reading pointers... 0: reading document index... 0: creating numpy buffer of mmap... 0: creating memory view of numpy buffer... 0: > finished creating indexed dataset in 0.082135 seconds 0: number of documents: 364608 0: > dataset split: 0: validation: 0: document indices in [0, 364608) total of 364608 documents 0: > loading doc-idx mapping from /scratch/project_462000119/data/c4_validation/gpt2tok_c4validation_rerun_text_document_validation_indexmap_51200ns_2048sl_1234s_doc_idx.npy 0: > loading sample-idx mapping from /scratch/project_462000119/data/c4_validation/gpt2tok_c4validation_rerun_text_document_validation_indexmap_51200ns_2048sl_1234s_sample_idx.npy 0: > loading shuffle-idx mapping from /scratch/project_462000119/data/c4_validation/gpt2tok_c4validation_rerun_text_document_validation_indexmap_51200ns_2048sl_1234s_shuffle_idx.npy 0: loaded indexed file in 0.123 seconds 0: total number of samples: 84978 0: total number of epochs: 1 0: > finished creating GPT datasets ... 0: [after dataloaders are built] datetime: 2023-05-13 12:05:38 0: done with setup ... 0: training ... 31: time (ms) | model-and-optimizer-setup: 57092.40 | train/valid/test-data-iterators-setup: 20063.58 0: [after training is done] datetime: 2023-05-13 12:05:38 31: ----------------------------------------------------------------------------------------------------------------- 31: validation loss at the end of training for val data | lm loss value: 2.659265E+00 | lm loss PPL: 1.428579E+01 | 31: ----------------------------------------------------------------------------------------------------------------- END 3511463: Sat 13 May 2023 12:06:34 PM EEST